2023-06-24T09:14:15.395 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-24T09:14:15.396 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893 --name yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi --verbose --description orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/jaeger 3-final} -- /home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893/orig.config.yaml 2023-06-24T09:14:15.426 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-24T09:14:15.511 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893 branch: reef description: orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/jaeger 3-final} email: yweinste@redhat.com first_in_suite: false job_id: '7314893' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: reef ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: reef sha1: b9d28de6006117997208584eb86d8d304f2a113d owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - host.a - client.0 - - host.b - client.1 seed: 3792 sha1: c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 sleep_before_teardown: 0 subset: 111/120000 suite: orch suite_branch: reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b9d28de6006117997208584eb86d8d304f2a113d/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b9d28de6006117997208584eb86d8d304f2a113d targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNBaO0t3/+SbaU6HMFkYX0kuNEYQm4jLZRjU6sCF0TOfNmkjFCbVqgn9nX1TpSuMGZ5vO2qqDLiNWNSOhhvv97w= smithi188.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD7mS1xfBOZDSTrkogxzQbG45xjsup+Y1b0h1In6iQjviqgcryJ+dyuLLMSAEMEPadkax4Q0EsbG2EVAtWdCVqZTxSodnboVwNrg4fg1tdiMtFLjFwXBoC3Q8iGNL4bGZsIbGyPyWBbTRkWNb6AKwF6YNvmQ7CramB1Wl4QSG6XztCAgnx1zD+2/afSzyLUNM9PK0F+ZiOxYRazSEkdj4YeSC0mWZ/2JbmFNNBhlt2tc/m+1odteQQZPz+R1oJ8Cz7+uYWilaeKFUYmxjZ98luem009g3HK0ffALugDsywZ+1Q1PPHWVF1ieSthmQaXUHe/JqBneGPJa7Wf7iWA7eFcx9e+wcv4jPHXZXY1n8FlqcO6uNXx+ivKvZubj31hmNGDPMiJoy4c5Vj2taNp+VCiShFia7lqElC8uULdXvTLAtg/lBxzwoN1CRHdTtrFGERs5W+nd6MyqWnxV4srLi2encB1DSqedtSYX/ON2vHBzaMN1H5DZqqYwklSlavwakk= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - sudo sed -i 's/runtime = "runc"/#runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/#runtime = "crun"/runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: roleless: true - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph orch apply jaeger - cephadm.wait_for_service: service: elasticsearch - cephadm.wait_for_service: service: jaeger-collector - cephadm.wait_for_service: service: jaeger-query - cephadm.wait_for_service: service: jaeger-agent - cephadm.shell: host.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls | grep '^osd.all-available-devices ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 076bbebc42a14f7d568aaa78eabb0038327bcb23 timestamp: 2023-06-23_20:59:31 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-24T09:14:15.512 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b9d28de6006117997208584eb86d8d304f2a113d/qa; will attempt to use it 2023-06-24T09:14:15.512 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b9d28de6006117997208584eb86d8d304f2a113d/qa/tasks 2023-06-24T09:14:15.513 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-24T09:14:15.514 INFO:teuthology.task.internal:Checking packages... 2023-06-24T09:14:15.544 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8' 2023-06-24T09:14:15.545 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-24T09:14:15.545 INFO:teuthology.packaging:ref: None 2023-06-24T09:14:15.545 INFO:teuthology.packaging:tag: None 2023-06-24T09:14:15.546 INFO:teuthology.packaging:branch: reef 2023-06-24T09:14:15.546 INFO:teuthology.packaging:sha1: c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 2023-06-24T09:14:15.546 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-06-24T09:14:15.798 INFO:teuthology.task.internal:Found packages for ceph version 18.1.0-176.g026e2477 2023-06-24T09:14:15.798 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-24T09:14:15.812 INFO:teuthology.task.internal:no buildpackages task found 2023-06-24T09:14:15.812 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-24T09:14:15.927 INFO:teuthology.task.internal:Saving configuration 2023-06-24T09:14:15.940 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-24T09:14:15.951 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-24T09:14:15.980 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-24 09:09:08.263027', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNBaO0t3/+SbaU6HMFkYX0kuNEYQm4jLZRjU6sCF0TOfNmkjFCbVqgn9nX1TpSuMGZ5vO2qqDLiNWNSOhhvv97w='} 2023-06-24T09:14:16.004 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-24 09:09:08.260994', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFGIXnauQLhOknwOy3xIjKyAeA9q/1+QC2DcEGRDzrVsNMyy3/Wmpv6wnD+hH4fZ9Bu5FWCjS3zhVcz3aiJ+ZZ4='} 2023-06-24T09:14:16.004 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-24T09:14:16.015 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['host.a', 'client.0'] 2023-06-24T09:14:16.016 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['host.b', 'client.1'] 2023-06-24T09:14:16.016 INFO:teuthology.run_tasks:Running task console_log... 2023-06-24T09:14:16.104 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f46fc047040>, signals=[15]) 2023-06-24T09:14:16.105 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-24T09:14:16.114 INFO:teuthology.task.internal:Opening connections... 2023-06-24T09:14:16.114 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2023-06-24T09:14:16.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-24T09:14:16.190 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2023-06-24T09:14:16.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-24T09:14:16.262 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-24T09:14:16.270 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2023-06-24T09:14:16.295 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2023-06-24T09:14:16.295 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2023-06-24T09:14:16.357 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2023-06-24T09:14:16.358 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8" 2023-06-24T09:14:16.358 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2023-06-24T09:14:16.358 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2023-06-24T09:14:16.358 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8" 2023-06-24T09:14:16.358 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8" 2023-06-24T09:14:16.358 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 8" 2023-06-24T09:14:16.359 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2023-06-24T09:14:16.359 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-06-24T09:14:16.359 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2023-06-24T09:14:16.359 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-24T09:14:16.359 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-24T09:14:16.359 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-06-24T09:14:16.360 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2023-06-24T09:14:16.381 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2023-06-24T09:14:16.405 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2023-06-24T09:14:16.405 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2023-06-24T09:14:16.465 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2023-06-24T09:14:16.465 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="8" 2023-06-24T09:14:16.466 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2023-06-24T09:14:16.466 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2023-06-24T09:14:16.466 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="8" 2023-06-24T09:14:16.466 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el8" 2023-06-24T09:14:16.466 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 8" 2023-06-24T09:14:16.466 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2023-06-24T09:14:16.466 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-06-24T09:14:16.467 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2023-06-24T09:14:16.467 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-24T09:14:16.467 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-24T09:14:16.467 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-06-24T09:14:16.467 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2023-06-24T09:14:16.490 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-24T09:14:16.504 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-24T09:14:16.515 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-24T09:14:16.517 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2023-06-24T09:14:16.519 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2023-06-24T09:14:16.539 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-24T09:14:16.544 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-24T09:14:16.545 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2023-06-24T09:14:16.580 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2023-06-24T09:14:16.622 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-24T09:14:16.682 INFO:teuthology.run_tasks:Running task kernel... 2023-06-24T09:14:16.699 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-06-24T09:14:16.699 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-06-24T09:14:16.700 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-24T09:14:16.701 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-24T09:14:16.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:14:16.722 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2023-06-24T09:14:16.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:14:16.723 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2023-06-24T09:14:16.782 INFO:teuthology.orchestra.run.smithi188.stdout:4.18.0-489.el8.x86_64 2023-06-24T09:14:16.782 INFO:teuthology.task.kernel:Running kernel on smithi188: 4.18.0-489.el8.x86_64 2023-06-24T09:14:16.782 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum install -y kernel 2023-06-24T09:14:16.784 INFO:teuthology.orchestra.run.smithi045.stdout:4.18.0-489.el8.x86_64 2023-06-24T09:14:16.784 INFO:teuthology.task.kernel:Running kernel on smithi045: 4.18.0-489.el8.x86_64 2023-06-24T09:14:16.784 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2023-06-24T09:14:20.879 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - AppStream 18 MB/s | 30 MB 00:01 2023-06-24T09:14:21.214 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 8 - AppStream 16 MB/s | 30 MB 00:01 2023-06-24T09:14:27.547 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 8 - BaseOS 39 MB/s | 39 MB 00:01 2023-06-24T09:14:27.687 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - BaseOS 35 MB/s | 39 MB 00:01 2023-06-24T09:14:33.538 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 8 - Extras 57 kB/s | 18 kB 00:00 2023-06-24T09:14:33.987 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - Extras 106 kB/s | 18 kB 00:00 2023-06-24T09:14:34.247 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 8 - Extras common packages 12 kB/s | 6.6 kB 00:00 2023-06-24T09:14:34.571 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 8 - Extras common packages 16 kB/s | 6.6 kB 00:00 2023-06-24T09:14:36.873 INFO:teuthology.orchestra.run.smithi188.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-24T09:14:36.998 INFO:teuthology.orchestra.run.smithi045.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2023-06-24T09:14:37.959 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-06-24T09:14:39.590 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 16 MB 00:02 2023-06-24T09:14:41.682 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 385 kB/s | 24 kB 00:00 2023-06-24T09:14:43.548 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 424 kB/s | 24 kB 00:00 2023-06-24T09:14:44.805 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-06-24T09:14:44.806 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-06-24T09:14:45.034 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2023-06-24T09:14:45.034 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-06-24T09:14:45.035 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2023-06-24T09:14:45.035 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-06-24T09:14:45.035 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2023-06-24T09:14:45.035 INFO:teuthology.orchestra.run.smithi188.stdout: kernel x86_64 4.18.0-497.el8 baseos 9.6 M 2023-06-24T09:14:45.035 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2023-06-24T09:14:45.035 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-core x86_64 4.18.0-497.el8 baseos 42 M 2023-06-24T09:14:45.035 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules x86_64 4.18.0-497.el8 baseos 35 M 2023-06-24T09:14:45.036 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:14:45.036 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2023-06-24T09:14:45.036 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-06-24T09:14:45.036 INFO:teuthology.orchestra.run.smithi188.stdout:Install 3 Packages 2023-06-24T09:14:45.036 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:14:45.037 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 87 M 2023-06-24T09:14:45.037 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 95 M 2023-06-24T09:14:45.037 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2023-06-24T09:14:45.492 INFO:teuthology.orchestra.run.smithi188.stdout:(1/3): kernel-4.18.0-497.el8.x86_64.rpm 23 MB/s | 9.6 MB 00:00 2023-06-24T09:14:46.042 INFO:teuthology.orchestra.run.smithi188.stdout:(2/3): kernel-modules-4.18.0-497.el8.x86_64.rpm 36 MB/s | 35 MB 00:00 2023-06-24T09:14:46.450 INFO:teuthology.orchestra.run.smithi188.stdout:(3/3): kernel-core-4.18.0-497.el8.x86_64.rpm 31 MB/s | 42 MB 00:01 2023-06-24T09:14:46.451 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2023-06-24T09:14:46.451 INFO:teuthology.orchestra.run.smithi188.stdout:Total 62 MB/s | 87 MB 00:01 2023-06-24T09:14:46.846 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-06-24T09:14:46.847 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-06-24T09:14:47.076 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-06-24T09:14:47.077 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-06-24T09:14:47.077 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2023-06-24T09:14:47.077 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-06-24T09:14:47.078 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2023-06-24T09:14:47.078 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 4.18.0-497.el8 baseos 9.6 M 2023-06-24T09:14:47.078 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-06-24T09:14:47.078 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 4.18.0-497.el8 baseos 42 M 2023-06-24T09:14:47.078 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 4.18.0-497.el8 baseos 35 M 2023-06-24T09:14:47.078 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:14:47.078 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-06-24T09:14:47.079 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-06-24T09:14:47.079 INFO:teuthology.orchestra.run.smithi045.stdout:Install 3 Packages 2023-06-24T09:14:47.079 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:14:47.079 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 87 M 2023-06-24T09:14:47.080 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 95 M 2023-06-24T09:14:47.080 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-06-24T09:14:47.105 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2023-06-24T09:14:47.508 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2023-06-24T09:14:47.508 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2023-06-24T09:14:47.716 INFO:teuthology.orchestra.run.smithi045.stdout:(1/3): kernel-4.18.0-497.el8.x86_64.rpm 23 MB/s | 9.6 MB 00:00 2023-06-24T09:14:48.233 INFO:teuthology.orchestra.run.smithi045.stdout:(2/3): kernel-modules-4.18.0-497.el8.x86_64.rpm 37 MB/s | 35 MB 00:00 2023-06-24T09:14:48.641 INFO:teuthology.orchestra.run.smithi045.stdout:(3/3): kernel-core-4.18.0-497.el8.x86_64.rpm 31 MB/s | 42 MB 00:01 2023-06-24T09:14:48.642 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-06-24T09:14:48.642 INFO:teuthology.orchestra.run.smithi045.stdout:Total 56 MB/s | 87 MB 00:01 2023-06-24T09:14:48.704 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2023-06-24T09:14:48.724 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2023-06-24T09:14:49.335 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-06-24T09:14:49.772 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-06-24T09:14:49.773 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-06-24T09:14:51.038 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-06-24T09:14:51.054 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-06-24T09:14:51.492 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2023-06-24T09:14:51.983 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-core-4.18.0-497.el8.x86_64 1/3 2023-06-24T09:14:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-06-24T09:14:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-4.18.0-497.el8.x86_64 1/3 2023-06-24T09:14:55.538 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-4.18.0-497.el8.x86_64 1/3 2023-06-24T09:14:55.916 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-modules-4.18.0-497.el8.x86_64 2/3 2023-06-24T09:14:58.166 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-4.18.0-497.el8.x86_64 1/3 2023-06-24T09:14:58.582 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-4.18.0-497.el8.x86_64 2/3 2023-06-24T09:14:59.262 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-4.18.0-497.el8.x86_64 2/3 2023-06-24T09:14:59.709 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-4.18.0-497.el8.x86_64 3/3 2023-06-24T09:15:02.169 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-4.18.0-497.el8.x86_64 2/3 2023-06-24T09:15:02.464 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-4.18.0-497.el8.x86_64 3/3 2023-06-24T09:15:21.641 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-4.18.0-497.el8.x86_64 3/3 2023-06-24T09:15:21.664 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-4.18.0-497.el8.x86_64 3/3 2023-06-24T09:15:22.178 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-4.18.0-497.el8.x86_64 3/3 2023-06-24T09:15:22.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-4.18.0-497.el8.x86_64 1/3 2023-06-24T09:15:22.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-core-4.18.0-497.el8.x86_64 2/3 2023-06-24T09:15:22.820 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-modules-4.18.0-497.el8.x86_64 3/3 2023-06-24T09:15:22.820 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:15:22.821 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2023-06-24T09:15:22.821 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-4.18.0-497.el8.x86_64 kernel-core-4.18.0-497.el8.x86_64 2023-06-24T09:15:22.821 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-4.18.0-497.el8.x86_64 2023-06-24T09:15:22.821 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:15:22.821 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2023-06-24T09:15:22.956 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2023-06-24T09:15:23.164 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-4.18.0-497.el8.x86_64 2023-06-24T09:15:23.165 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-497.el8.x86_64 2023-06-24T09:15:23.165 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-497.el8.x86_64 2023-06-24T09:15:23.165 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-24T09:15:23.165 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-24T09:15:23.166 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2023-06-24T09:15:23.166 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi188.front.sepia.ceph.com, path=None, version=distro) 2023-06-24T09:15:23.166 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2023-06-24T09:15:23.373 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-4.18.0-497.el8.x86_64 2023-06-24T09:15:23.373 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-497.el8.x86_64 2023-06-24T09:15:23.374 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -qi grub2-tools 2023-06-24T09:15:23.407 INFO:teuthology.orchestra.run.smithi188.stdout:Name : grub2-tools 2023-06-24T09:15:23.407 INFO:teuthology.orchestra.run.smithi188.stdout:Epoch : 1 2023-06-24T09:15:23.408 INFO:teuthology.orchestra.run.smithi188.stdout:Version : 2.02 2023-06-24T09:15:23.408 INFO:teuthology.orchestra.run.smithi188.stdout:Release : 129.el8 2023-06-24T09:15:23.408 INFO:teuthology.orchestra.run.smithi188.stdout:Architecture: x86_64 2023-06-24T09:15:23.408 INFO:teuthology.orchestra.run.smithi188.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-06-24T09:15:23.408 INFO:teuthology.orchestra.run.smithi188.stdout:Group : System Environment/Base 2023-06-24T09:15:23.408 INFO:teuthology.orchestra.run.smithi188.stdout:Size : 9265586 2023-06-24T09:15:23.409 INFO:teuthology.orchestra.run.smithi188.stdout:License : GPLv3+ 2023-06-24T09:15:23.409 INFO:teuthology.orchestra.run.smithi188.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-06-24T09:15:23.409 INFO:teuthology.orchestra.run.smithi188.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-06-24T09:15:23.409 INFO:teuthology.orchestra.run.smithi188.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-06-24T09:15:23.409 INFO:teuthology.orchestra.run.smithi188.stdout:Build Host : kbuilder.bsys.centos.org 2023-06-24T09:15:23.409 INFO:teuthology.orchestra.run.smithi188.stdout:Relocations : (not relocatable) 2023-06-24T09:15:23.409 INFO:teuthology.orchestra.run.smithi188.stdout:Packager : CentOS BuildSystem 2023-06-24T09:15:23.410 INFO:teuthology.orchestra.run.smithi188.stdout:Vendor : CentOS 2023-06-24T09:15:23.410 INFO:teuthology.orchestra.run.smithi188.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-24T09:15:23.410 INFO:teuthology.orchestra.run.smithi188.stdout:Summary : Support tools for GRUB. 2023-06-24T09:15:23.410 INFO:teuthology.orchestra.run.smithi188.stdout:Description : 2023-06-24T09:15:23.410 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:15:23.410 INFO:teuthology.orchestra.run.smithi188.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-24T09:15:23.410 INFO:teuthology.orchestra.run.smithi188.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-24T09:15:23.410 INFO:teuthology.orchestra.run.smithi188.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-24T09:15:23.411 INFO:teuthology.orchestra.run.smithi188.stdout:hardware devices. 2023-06-24T09:15:23.411 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:15:23.411 INFO:teuthology.orchestra.run.smithi188.stdout:This subpackage provides tools for support of all platforms. 2023-06-24T09:15:23.412 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-24T09:15:23.412 INFO:teuthology.task.kernel:Updating grub on smithi188 to boot 4.18.0-497.el8.x86_64 2023-06-24T09:15:23.412 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-24T09:15:24.808 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2023-06-24T09:15:25.652 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-4.18.0-497.el8.x86_64 3/3 2023-06-24T09:15:25.675 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-4.18.0-497.el8.x86_64 3/3 2023-06-24T09:15:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-4.18.0-497.el8.x86_64 3/3 2023-06-24T09:15:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-4.18.0-497.el8.x86_64 1/3 2023-06-24T09:15:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-4.18.0-497.el8.x86_64 2/3 2023-06-24T09:15:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-4.18.0-497.el8.x86_64 3/3 2023-06-24T09:15:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:15:26.755 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-06-24T09:15:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-4.18.0-497.el8.x86_64 kernel-core-4.18.0-497.el8.x86_64 2023-06-24T09:15:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-4.18.0-497.el8.x86_64 2023-06-24T09:15:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:15:26.755 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-06-24T09:15:26.960 INFO:teuthology.orchestra.run.smithi188.stderr:done 2023-06-24T09:15:26.962 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2023-06-24T09:15:27.013 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.V6E2d9x1TX 2023-06-24T09:15:27.014 DEBUG:teuthology.orchestra.run.smithi188:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.V6E2d9x1TX 2023-06-24T09:15:27.047 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.V6E2d9x1TX 2023-06-24T09:15:27.191 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.V6E2d9x1TX is 6KB 2023-06-24T09:15:27.239 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.V6E2d9x1TX 2023-06-24T09:15:27.256 DEBUG:teuthology.orchestra.run.smithi188:> sudo /bin/ls /boot/loader/entries || true 2023-06-24T09:15:27.329 INFO:teuthology.orchestra.run.smithi188.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-06-24T09:15:27.329 INFO:teuthology.orchestra.run.smithi188.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-06-24T09:15:27.330 INFO:teuthology.orchestra.run.smithi188.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-06-24T09:15:27.330 INFO:teuthology.orchestra.run.smithi188.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-497.el8.x86_64.conf 2023-06-24T09:15:27.331 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-497.el8.x86_64 2023-06-24T09:15:27.460 DEBUG:teuthology.orchestra.run.smithi188:> sudo shutdown -r now 2023-06-24T09:15:27.775 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2023-06-24T09:15:28.010 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-4.18.0-497.el8.x86_64 2023-06-24T09:15:28.011 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-497.el8.x86_64 2023-06-24T09:15:28.011 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-497.el8.x86_64 2023-06-24T09:15:28.011 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-24T09:15:28.011 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-24T09:15:28.011 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2023-06-24T09:15:28.012 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2023-06-24T09:15:28.012 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2023-06-24T09:15:28.227 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-4.18.0-497.el8.x86_64 2023-06-24T09:15:28.228 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-497.el8.x86_64 2023-06-24T09:15:28.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2023-06-24T09:15:28.262 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2023-06-24T09:15:28.263 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2023-06-24T09:15:28.263 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.02 2023-06-24T09:15:28.263 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 129.el8 2023-06-24T09:15:28.263 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2023-06-24T09:15:28.263 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-06-24T09:15:28.263 INFO:teuthology.orchestra.run.smithi045.stdout:Group : System Environment/Base 2023-06-24T09:15:28.263 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 9265586 2023-06-24T09:15:28.264 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2023-06-24T09:15:28.264 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-06-24T09:15:28.264 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-06-24T09:15:28.264 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-06-24T09:15:28.264 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : kbuilder.bsys.centos.org 2023-06-24T09:15:28.264 INFO:teuthology.orchestra.run.smithi045.stdout:Relocations : (not relocatable) 2023-06-24T09:15:28.264 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : CentOS BuildSystem 2023-06-24T09:15:28.265 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2023-06-24T09:15:28.265 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-24T09:15:28.265 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2023-06-24T09:15:28.265 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2023-06-24T09:15:28.265 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:15:28.265 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-24T09:15:28.266 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-24T09:15:28.266 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-24T09:15:28.266 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2023-06-24T09:15:28.266 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:15:28.266 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2023-06-24T09:15:28.267 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-24T09:15:28.267 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 4.18.0-497.el8.x86_64 2023-06-24T09:15:28.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-24T09:15:29.245 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2023-06-24T09:15:31.296 INFO:teuthology.orchestra.run.smithi045.stderr:done 2023-06-24T09:15:31.298 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2023-06-24T09:15:31.319 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.2alcmMEjYk 2023-06-24T09:15:31.319 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2alcmMEjYk 2023-06-24T09:15:31.400 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.2alcmMEjYk 2023-06-24T09:15:31.540 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.2alcmMEjYk is 6KB 2023-06-24T09:15:31.588 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.2alcmMEjYk 2023-06-24T09:15:31.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2023-06-24T09:15:31.675 INFO:teuthology.orchestra.run.smithi045.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-06-24T09:15:31.676 INFO:teuthology.orchestra.run.smithi045.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-06-24T09:15:31.676 INFO:teuthology.orchestra.run.smithi045.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-06-24T09:15:31.676 INFO:teuthology.orchestra.run.smithi045.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-497.el8.x86_64.conf 2023-06-24T09:15:31.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-497.el8.x86_64 2023-06-24T09:15:31.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2023-06-24T09:15:57.488 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2023-06-24T09:15:57.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-06-24T09:15:57.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-24T09:16:01.860 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2023-06-24T09:16:01.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-06-24T09:16:01.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-24T09:16:15.860 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-06-24T09:16:20.208 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-06-24T09:16:45.884 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-06-24T09:16:45.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-24T09:16:50.212 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-06-24T09:16:50.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-24T09:17:04.240 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-06-24T09:17:06.400 DEBUG:teuthology.orchestra.run.smithi045:> true 2023-06-24T09:17:06.778 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-06-24T09:17:06.778 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-497.el8.x86_64"... 2023-06-24T09:17:06.778 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2023-06-24T09:17:06.798 INFO:teuthology.orchestra.run.smithi045.stdout:4.18.0-497.el8.x86_64 2023-06-24T09:17:06.798 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-497.el8.x86_64 vs 4.18.0-497.el8.x86_64 2023-06-24T09:17:06.799 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-24T09:17:06.799 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-06-24T09:17:07.800 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2023-06-24T09:17:07.800 DEBUG:teuthology.orchestra.run.smithi045:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-24T09:17:07.876 INFO:teuthology.orchestra.run.smithi045.stdout:ttyS1 2023-06-24T09:17:07.915 DEBUG:teuthology.parallel:result is None 2023-06-24T09:17:34.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-06-24T09:17:34.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-24T09:17:34.535 DEBUG:teuthology.orchestra.run.smithi188:> true 2023-06-24T09:17:34.805 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-06-24T09:17:34.805 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-497.el8.x86_64"... 2023-06-24T09:17:34.806 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2023-06-24T09:17:34.824 INFO:teuthology.orchestra.run.smithi188.stdout:4.18.0-497.el8.x86_64 2023-06-24T09:17:34.825 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-497.el8.x86_64 vs 4.18.0-497.el8.x86_64 2023-06-24T09:17:34.825 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-24T09:17:34.825 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-06-24T09:17:35.826 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2023-06-24T09:17:35.827 DEBUG:teuthology.orchestra.run.smithi188:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-24T09:17:35.892 INFO:teuthology.orchestra.run.smithi188.stdout:ttyS1 2023-06-24T09:17:35.905 DEBUG:teuthology.parallel:result is None 2023-06-24T09:17:35.905 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-24T09:17:35.913 INFO:teuthology.task.internal:Creating test directory... 2023-06-24T09:17:35.913 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-24T09:17:35.917 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-24T09:17:35.939 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-24T09:17:35.946 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-24T09:17:35.957 INFO:teuthology.task.internal:Creating archive directory... 2023-06-24T09:17:35.957 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-24T09:17:35.982 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-24T09:17:36.013 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-24T09:17:36.020 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-24T09:17:36.021 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-24T09:17:36.050 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-24T09:17:36.087 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-24T09:17:36.088 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-24T09:17:36.099 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-24T09:17:36.099 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-24T09:17:36.101 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-24T09:17:36.109 INFO:teuthology.task.internal:Configuring sudo... 2023-06-24T09:17:36.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-24T09:17:36.143 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-24T09:17:36.173 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-24T09:17:36.182 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-24T09:17:36.183 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-24T09:17:36.215 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-24T09:17:36.234 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-24T09:17:36.324 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-24T09:17:36.375 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-24T09:17:36.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-24T09:17:36.505 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:17:36.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-24T09:17:36.574 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-24T09:17:36.619 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-24T09:17:36.676 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-24T09:17:36.765 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-24T09:17:36.803 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-06-24T09:17:36.804 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-24T09:17:36.870 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2023-06-24T09:17:36.872 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2023-06-24T09:17:36.948 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-24T09:17:36.969 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-24T09:17:37.444 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-24T09:17:37.453 INFO:teuthology.task.internal:Starting timer... 2023-06-24T09:17:37.453 INFO:teuthology.run_tasks:Running task pcp... 2023-06-24T09:17:37.469 INFO:teuthology.run_tasks:Running task selinux... 2023-06-24T09:17:37.481 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-06-24T09:17:37.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2023-06-24T09:17:37.578 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2023-06-24T09:17:37.580 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2023-06-24T09:17:37.661 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2023-06-24T09:17:37.662 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-24T09:17:37.663 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2023-06-24T09:17:37.688 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2023-06-24T09:17:37.689 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2023-06-24T09:17:37.725 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2023-06-24T09:17:37.725 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2023-06-24T09:17:37.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-24T09:17:37.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:17:37.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-24T09:17:37.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:17:37.794 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-24T09:17:37.794 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2023-06-24T09:17:37.828 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2023-06-24T09:17:37.863 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-24T09:17:37.875 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-06-24T09:17:37.882 INFO:teuthology.repo_utils:Fetching main from origin 2023-06-24T09:17:38.017 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-24T09:17:38.044 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-24T09:17:38.046 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-24T09:24:40.598 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2023-06-24T09:24:40.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-06-24T09:24:40.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-24T09:24:40.672 DEBUG:teuthology.orchestra.run.smithi045:> true 2023-06-24T09:24:40.699 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-06-24T09:24:40.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-24T09:24:40.769 DEBUG:teuthology.orchestra.run.smithi188:> true 2023-06-24T09:24:40.843 INFO:teuthology.run_tasks:Running task clock... 2023-06-24T09:24:40.858 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-24T09:24:40.859 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-24T09:24:40.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-24T09:24:40.862 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-24T09:24:40.862 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-24T09:24:40.895 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-24T09:24:40.911 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-24T09:24:40.916 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-24T09:24:40.930 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-24T09:24:40.986 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2023-06-24T09:24:40.991 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2023-06-24T09:24:40.999 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2023-06-24T09:24:41.002 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2023-06-24T09:24:41.013 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-24T09:24:41.016 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-24T09:24:41.028 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-24T09:24:41.029 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-24T09:24:41.106 INFO:teuthology.orchestra.run.smithi188.stderr:bash: ntpq: command not found 2023-06-24T09:24:41.109 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-24T09:24:41.110 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2023-06-24T09:24:41.110 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-24T09:24:41.110 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-24T09:24:41.110 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-24T09:24:41.110 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-24T09:24:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:bash: ntpq: command not found 2023-06-24T09:24:41.163 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-24T09:24:41.163 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2023-06-24T09:24:41.164 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-24T09:24:41.164 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-24T09:24:41.164 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-24T09:24:41.164 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-24T09:24:41.165 INFO:teuthology.run_tasks:Running task pexec... 2023-06-24T09:24:41.176 INFO:teuthology.task.pexec:Executing custom commands... 2023-06-24T09:24:41.177 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi045.front.sepia.ceph.com 2023-06-24T09:24:41.177 DEBUG:teuthology.orchestra.run.smithi045:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-06-24T09:24:41.178 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi188.front.sepia.ceph.com 2023-06-24T09:24:41.178 DEBUG:teuthology.orchestra.run.smithi188:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-06-24T09:24:41.999 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:28 ago on Sat 24 Jun 2023 09:23:13 AM UTC. 2023-06-24T09:24:42.061 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:19 ago on Sat 24 Jun 2023 09:23:22 AM UTC. 2023-06-24T09:24:43.599 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2023-06-24T09:24:43.600 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-06-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2023-06-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-06-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi188.stdout:Resetting modules: 2023-06-24T09:24:43.601 INFO:teuthology.orchestra.run.smithi188.stdout: container-tools 2023-06-24T09:24:43.602 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:24:43.602 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2023-06-24T09:24:43.603 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-06-24T09:24:43.603 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:24:43.782 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-06-24T09:24:43.783 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-06-24T09:24:43.784 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2023-06-24T09:24:43.784 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-06-24T09:24:43.784 INFO:teuthology.orchestra.run.smithi045.stdout:Resetting modules: 2023-06-24T09:24:43.784 INFO:teuthology.orchestra.run.smithi045.stdout: container-tools 2023-06-24T09:24:43.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:24:43.784 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-06-24T09:24:43.785 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-06-24T09:24:43.785 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:24:43.869 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2023-06-24T09:24:44.100 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-06-24T09:24:44.741 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:01:31 ago on Sat 24 Jun 2023 09:23:13 AM UTC. 2023-06-24T09:24:45.004 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:22 ago on Sat 24 Jun 2023 09:23:22 AM UTC. 2023-06-24T09:24:46.359 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2023-06-24T09:24:46.480 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-06-24T09:24:46.481 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repo Size 2023-06-24T09:24:46.481 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-06-24T09:24:46.481 INFO:teuthology.orchestra.run.smithi188.stdout:Installing group/module packages: 2023-06-24T09:24:46.481 INFO:teuthology.orchestra.run.smithi188.stdout: aardvark-dns x86_64 2:1.4.0-1.module_el8.8.0+1254+78119b6e appstream 986 k 2023-06-24T09:24:46.481 INFO:teuthology.orchestra.run.smithi188.stdout: buildah x86_64 1:1.28.2-2.module_el8.8.0+1254+78119b6e appstream 8.1 M 2023-06-24T09:24:46.482 INFO:teuthology.orchestra.run.smithi188.stdout: cockpit-podman noarch 59-1.module_el8.8.0+1254+78119b6e appstream 579 k 2023-06-24T09:24:46.482 INFO:teuthology.orchestra.run.smithi188.stdout: crun x86_64 1.7.2-1.module_el8.8.0+1254+78119b6e appstream 218 k 2023-06-24T09:24:46.482 INFO:teuthology.orchestra.run.smithi188.stdout: netavark x86_64 2:1.4.0-1.module_el8.8.0+1254+78119b6e appstream 1.8 M 2023-06-24T09:24:46.482 INFO:teuthology.orchestra.run.smithi188.stdout: python3-podman noarch 4.3.0-2.module_el8.8.0+1254+78119b6e appstream 152 k 2023-06-24T09:24:46.482 INFO:teuthology.orchestra.run.smithi188.stdout: skopeo x86_64 2:1.10.0-1.module_el8.8.0+1254+78119b6e appstream 6.8 M 2023-06-24T09:24:46.482 INFO:teuthology.orchestra.run.smithi188.stdout: toolbox x86_64 0.0.99.3-5.module_el8.8.0+1254+78119b6e appstream 2.4 M 2023-06-24T09:24:46.482 INFO:teuthology.orchestra.run.smithi188.stdout: udica noarch 0.2.6-4.module_el8.8.0+1254+78119b6e appstream 49 k 2023-06-24T09:24:46.482 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2023-06-24T09:24:46.483 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2023-06-24T09:24:46.483 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2023-06-24T09:24:46.483 INFO:teuthology.orchestra.run.smithi188.stdout: yajl x86_64 2.1.0-11.el8 appstream 41 k 2023-06-24T09:24:46.483 INFO:teuthology.orchestra.run.smithi188.stdout:Installing module profiles: 2023-06-24T09:24:46.483 INFO:teuthology.orchestra.run.smithi188.stdout: container-tools/common 2023-06-24T09:24:46.483 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:24:46.483 INFO:teuthology.orchestra.run.smithi188.stdout:Enabling module streams: 2023-06-24T09:24:46.484 INFO:teuthology.orchestra.run.smithi188.stdout: container-tools 2023-06-24T09:24:46.484 INFO:teuthology.orchestra.run.smithi188.stdout: rhel8 2023-06-24T09:24:46.484 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:24:46.484 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2023-06-24T09:24:46.484 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-06-24T09:24:46.484 INFO:teuthology.orchestra.run.smithi188.stdout:Install 12 Packages 2023-06-24T09:24:46.484 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:24:46.485 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 21 M 2023-06-24T09:24:46.485 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 68 M 2023-06-24T09:24:46.485 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2023-06-24T09:24:46.707 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2023-06-24T09:24:46.839 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-06-24T09:24:46.839 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2023-06-24T09:24:46.839 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-06-24T09:24:46.840 INFO:teuthology.orchestra.run.smithi045.stdout:Installing group/module packages: 2023-06-24T09:24:46.840 INFO:teuthology.orchestra.run.smithi045.stdout: aardvark-dns x86_64 2:1.4.0-1.module_el8.8.0+1254+78119b6e appstream 986 k 2023-06-24T09:24:46.840 INFO:teuthology.orchestra.run.smithi045.stdout: buildah x86_64 1:1.28.2-2.module_el8.8.0+1254+78119b6e appstream 8.1 M 2023-06-24T09:24:46.840 INFO:teuthology.orchestra.run.smithi045.stdout: cockpit-podman noarch 59-1.module_el8.8.0+1254+78119b6e appstream 579 k 2023-06-24T09:24:46.840 INFO:teuthology.orchestra.run.smithi045.stdout: crun x86_64 1.7.2-1.module_el8.8.0+1254+78119b6e appstream 218 k 2023-06-24T09:24:46.840 INFO:teuthology.orchestra.run.smithi045.stdout: netavark x86_64 2:1.4.0-1.module_el8.8.0+1254+78119b6e appstream 1.8 M 2023-06-24T09:24:46.840 INFO:teuthology.orchestra.run.smithi045.stdout: python3-podman noarch 4.3.0-2.module_el8.8.0+1254+78119b6e appstream 152 k 2023-06-24T09:24:46.841 INFO:teuthology.orchestra.run.smithi045.stdout: skopeo x86_64 2:1.10.0-1.module_el8.8.0+1254+78119b6e appstream 6.8 M 2023-06-24T09:24:46.841 INFO:teuthology.orchestra.run.smithi045.stdout: toolbox x86_64 0.0.99.3-5.module_el8.8.0+1254+78119b6e appstream 2.4 M 2023-06-24T09:24:46.841 INFO:teuthology.orchestra.run.smithi045.stdout: udica noarch 0.2.6-4.module_el8.8.0+1254+78119b6e appstream 49 k 2023-06-24T09:24:46.841 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2023-06-24T09:24:46.841 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2023-06-24T09:24:46.841 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2023-06-24T09:24:46.841 INFO:teuthology.orchestra.run.smithi045.stdout: yajl x86_64 2.1.0-11.el8 appstream 41 k 2023-06-24T09:24:46.842 INFO:teuthology.orchestra.run.smithi045.stdout:Installing module profiles: 2023-06-24T09:24:46.842 INFO:teuthology.orchestra.run.smithi045.stdout: container-tools/common 2023-06-24T09:24:46.842 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:24:46.842 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling module streams: 2023-06-24T09:24:46.842 INFO:teuthology.orchestra.run.smithi045.stdout: container-tools 2023-06-24T09:24:46.842 INFO:teuthology.orchestra.run.smithi045.stdout: rhel8 2023-06-24T09:24:46.842 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:24:46.842 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2023-06-24T09:24:46.843 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2023-06-24T09:24:46.843 INFO:teuthology.orchestra.run.smithi045.stdout:Install 12 Packages 2023-06-24T09:24:46.843 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:24:46.843 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 21 M 2023-06-24T09:24:46.844 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 68 M 2023-06-24T09:24:46.844 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2023-06-24T09:24:46.961 INFO:teuthology.orchestra.run.smithi188.stdout:(1/12): cockpit-podman-59-1.module_el8.8.0+1254 2.0 MB/s | 579 kB 00:00 2023-06-24T09:24:47.003 INFO:teuthology.orchestra.run.smithi188.stdout:(2/12): aardvark-dns-1.4.0-1.module_el8.8.0+125 2.9 MB/s | 986 kB 00:00 2023-06-24T09:24:47.045 INFO:teuthology.orchestra.run.smithi188.stdout:(3/12): crun-1.7.2-1.module_el8.8.0+1254+78119b 2.6 MB/s | 218 kB 00:00 2023-06-24T09:24:47.112 INFO:teuthology.orchestra.run.smithi188.stdout:(4/12): python3-podman-4.3.0-2.module_el8.8.0+1 2.2 MB/s | 152 kB 00:00 2023-06-24T09:24:47.154 INFO:teuthology.orchestra.run.smithi188.stdout:(5/12): netavark-1.4.0-1.module_el8.8.0+1254+78 12 MB/s | 1.8 MB 00:00 2023-06-24T09:24:47.171 INFO:teuthology.orchestra.run.smithi188.stdout:(6/12): python3-pytoml-0.1.14-5.git7dea353.el8. 422 kB/s | 25 kB 00:00 2023-06-24T09:24:47.174 INFO:teuthology.orchestra.run.smithi045.stdout:(1/12): cockpit-podman-59-1.module_el8.8.0+1254 2.0 MB/s | 579 kB 00:00 2023-06-24T09:24:47.213 INFO:teuthology.orchestra.run.smithi188.stdout:(7/12): python3-pyxdg-0.25-16.el8.noarch.rpm 1.6 MB/s | 94 kB 00:00 2023-06-24T09:24:47.216 INFO:teuthology.orchestra.run.smithi045.stdout:(2/12): aardvark-dns-1.4.0-1.module_el8.8.0+125 2.9 MB/s | 986 kB 00:00 2023-06-24T09:24:47.261 INFO:teuthology.orchestra.run.smithi045.stdout:(3/12): crun-1.7.2-1.module_el8.8.0+1254+78119b 2.5 MB/s | 218 kB 00:00 2023-06-24T09:24:47.342 INFO:teuthology.orchestra.run.smithi045.stdout:(4/12): python3-podman-4.3.0-2.module_el8.8.0+1 1.8 MB/s | 152 kB 00:00 2023-06-24T09:24:47.355 INFO:teuthology.orchestra.run.smithi188.stdout:(8/12): toolbox-0.0.99.3-5.module_el8.8.0+1254+ 17 MB/s | 2.4 MB 00:00 2023-06-24T09:24:47.384 INFO:teuthology.orchestra.run.smithi045.stdout:(5/12): netavark-1.4.0-1.module_el8.8.0+1254+78 11 MB/s | 1.8 MB 00:00 2023-06-24T09:24:47.409 INFO:teuthology.orchestra.run.smithi045.stdout:(6/12): python3-pytoml-0.1.14-5.git7dea353.el8. 369 kB/s | 25 kB 00:00 2023-06-24T09:24:47.435 INFO:teuthology.orchestra.run.smithi045.stdout:(7/12): python3-pyxdg-0.25-16.el8.noarch.rpm 1.8 MB/s | 94 kB 00:00 2023-06-24T09:24:47.455 INFO:teuthology.orchestra.run.smithi188.stdout:(9/12): buildah-1.28.2-2.module_el8.8.0+1254+78 10 MB/s | 8.1 MB 00:00 2023-06-24T09:24:47.472 INFO:teuthology.orchestra.run.smithi188.stdout:(10/12): udica-0.2.6-4.module_el8.8.0+1254+7811 421 kB/s | 49 kB 00:00 2023-06-24T09:24:47.539 INFO:teuthology.orchestra.run.smithi188.stdout:(11/12): yajl-2.1.0-11.el8.x86_64.rpm 488 kB/s | 41 kB 00:00 2023-06-24T09:24:47.585 INFO:teuthology.orchestra.run.smithi045.stdout:(8/12): toolbox-0.0.99.3-5.module_el8.8.0+1254+ 16 MB/s | 2.4 MB 00:00 2023-06-24T09:24:47.623 INFO:teuthology.orchestra.run.smithi188.stdout:(12/12): skopeo-1.10.0-1.module_el8.8.0+1254+78 15 MB/s | 6.8 MB 00:00 2023-06-24T09:24:47.623 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2023-06-24T09:24:47.624 INFO:teuthology.orchestra.run.smithi188.stdout:Total 19 MB/s | 21 MB 00:01 2023-06-24T09:24:47.644 INFO:teuthology.orchestra.run.smithi045.stdout:(9/12): udica-0.2.6-4.module_el8.8.0+1254+78119 843 kB/s | 49 kB 00:00 2023-06-24T09:24:47.702 INFO:teuthology.orchestra.run.smithi045.stdout:(10/12): yajl-2.1.0-11.el8.x86_64.rpm 701 kB/s | 41 kB 00:00 2023-06-24T09:24:47.850 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2023-06-24T09:24:47.869 INFO:teuthology.orchestra.run.smithi045.stdout:(11/12): buildah-1.28.2-2.module_el8.8.0+1254+7 8.3 MB/s | 8.1 MB 00:00 2023-06-24T09:24:47.870 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2023-06-24T09:24:47.872 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2023-06-24T09:24:47.986 INFO:teuthology.orchestra.run.smithi045.stdout:(12/12): skopeo-1.10.0-1.module_el8.8.0+1254+78 12 MB/s | 6.8 MB 00:00 2023-06-24T09:24:47.986 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-06-24T09:24:47.987 INFO:teuthology.orchestra.run.smithi045.stdout:Total 18 MB/s | 21 MB 00:01 2023-06-24T09:24:48.086 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2023-06-24T09:24:48.090 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2023-06-24T09:24:48.220 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2023-06-24T09:24:48.241 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2023-06-24T09:24:48.241 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2023-06-24T09:24:48.533 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2023-06-24T09:24:48.536 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2023-06-24T09:24:48.667 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2023-06-24T09:24:48.869 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : yajl-2.1.0-11.el8.x86_64 1/12 2023-06-24T09:24:49.062 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 2/12 2023-06-24T09:24:49.156 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2023-06-24T09:24:49.327 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : yajl-2.1.0-11.el8.x86_64 1/12 2023-06-24T09:24:49.328 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/12 2023-06-24T09:24:49.513 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 2/12 2023-06-24T09:24:49.699 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : aardvark-dns-2:1.4.0-1.module_el8.8.0+1254+78119b6 4/12 2023-06-24T09:24:49.821 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/12 2023-06-24T09:24:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : aardvark-dns-2:1.4.0-1.module_el8.8.0+1254+78119b6 4/12 2023-06-24T09:24:50.568 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : netavark-2:1.4.0-1.module_el8.8.0+1254+78119b6e.x8 5/12 2023-06-24T09:24:50.814 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : buildah-1:1.28.2-2.module_el8.8.0+1254+78119b6e.x8 6/12 2023-06-24T09:24:51.105 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-podman-4.3.0-2.module_el8.8.0+1254+78119b6 7/12 2023-06-24T09:24:51.259 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : crun-1.7.2-1.module_el8.8.0+1254+78119b6e.x86_64 8/12 2023-06-24T09:24:51.340 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : netavark-2:1.4.0-1.module_el8.8.0+1254+78119b6e.x8 5/12 2023-06-24T09:24:51.600 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : udica-0.2.6-4.module_el8.8.0+1254+78119b6e.noarch 9/12 2023-06-24T09:24:51.629 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : buildah-1:1.28.2-2.module_el8.8.0+1254+78119b6e.x8 6/12 2023-06-24T09:24:51.884 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-podman-4.3.0-2.module_el8.8.0+1254+78119b6 7/12 2023-06-24T09:24:52.033 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : crun-1.7.2-1.module_el8.8.0+1254+78119b6e.x86_64 8/12 2023-06-24T09:24:52.356 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : toolbox-0.0.99.3-5.module_el8.8.0+1254+78119b6e.x8 10/12 2023-06-24T09:24:52.421 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : udica-0.2.6-4.module_el8.8.0+1254+78119b6e.noarch 9/12 2023-06-24T09:24:52.580 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : skopeo-2:1.10.0-1.module_el8.8.0+1254+78119b6e.x86 11/12 2023-06-24T09:24:52.777 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cockpit-podman-59-1.module_el8.8.0+1254+78119b6e.n 12/12 2023-06-24T09:24:53.105 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cockpit-podman-59-1.module_el8.8.0+1254+78119b6e.n 12/12 2023-06-24T09:24:53.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : aardvark-dns-2:1.4.0-1.module_el8.8.0+1254+78119b6 1/12 2023-06-24T09:24:53.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : buildah-1:1.28.2-2.module_el8.8.0+1254+78119b6e.x8 2/12 2023-06-24T09:24:53.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cockpit-podman-59-1.module_el8.8.0+1254+78119b6e.n 3/12 2023-06-24T09:24:53.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : crun-1.7.2-1.module_el8.8.0+1254+78119b6e.x86_64 4/12 2023-06-24T09:24:53.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : netavark-2:1.4.0-1.module_el8.8.0+1254+78119b6e.x8 5/12 2023-06-24T09:24:53.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-podman-4.3.0-2.module_el8.8.0+1254+78119b6 6/12 2023-06-24T09:24:53.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 7/12 2023-06-24T09:24:53.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 8/12 2023-06-24T09:24:53.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : skopeo-2:1.10.0-1.module_el8.8.0+1254+78119b6e.x86 9/12 2023-06-24T09:24:53.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : toolbox-0.0.99.3-5.module_el8.8.0+1254+78119b6e.x8 10/12 2023-06-24T09:24:53.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : udica-0.2.6-4.module_el8.8.0+1254+78119b6e.noarch 11/12 2023-06-24T09:24:53.173 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : toolbox-0.0.99.3-5.module_el8.8.0+1254+78119b6e.x8 10/12 2023-06-24T09:24:53.392 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : skopeo-2:1.10.0-1.module_el8.8.0+1254+78119b6e.x86 11/12 2023-06-24T09:24:53.619 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cockpit-podman-59-1.module_el8.8.0+1254+78119b6e.n 12/12 2023-06-24T09:24:53.754 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : yajl-2.1.0-11.el8.x86_64 12/12 2023-06-24T09:24:53.754 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:24:53.754 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2023-06-24T09:24:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: aardvark-dns-2:1.4.0-1.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: buildah-1:1.28.2-2.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: cockpit-podman-59-1.module_el8.8.0+1254+78119b6e.noarch 2023-06-24T09:24:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: crun-1.7.2-1.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: netavark-2:1.4.0-1.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: python3-podman-4.3.0-2.module_el8.8.0+1254+78119b6e.noarch 2023-06-24T09:24:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-06-24T09:24:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-06-24T09:24:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: skopeo-2:1.10.0-1.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: toolbox-0.0.99.3-5.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: udica-0.2.6-4.module_el8.8.0+1254+78119b6e.noarch 2023-06-24T09:24:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: yajl-2.1.0-11.el8.x86_64 2023-06-24T09:24:53.758 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:24:53.758 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2023-06-24T09:24:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cockpit-podman-59-1.module_el8.8.0+1254+78119b6e.n 12/12 2023-06-24T09:24:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : aardvark-dns-2:1.4.0-1.module_el8.8.0+1254+78119b6 1/12 2023-06-24T09:24:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : buildah-1:1.28.2-2.module_el8.8.0+1254+78119b6e.x8 2/12 2023-06-24T09:24:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cockpit-podman-59-1.module_el8.8.0+1254+78119b6e.n 3/12 2023-06-24T09:24:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : crun-1.7.2-1.module_el8.8.0+1254+78119b6e.x86_64 4/12 2023-06-24T09:24:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : netavark-2:1.4.0-1.module_el8.8.0+1254+78119b6e.x8 5/12 2023-06-24T09:24:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-podman-4.3.0-2.module_el8.8.0+1254+78119b6 6/12 2023-06-24T09:24:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 7/12 2023-06-24T09:24:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 8/12 2023-06-24T09:24:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : skopeo-2:1.10.0-1.module_el8.8.0+1254+78119b6e.x86 9/12 2023-06-24T09:24:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : toolbox-0.0.99.3-5.module_el8.8.0+1254+78119b6e.x8 10/12 2023-06-24T09:24:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : udica-0.2.6-4.module_el8.8.0+1254+78119b6e.noarch 11/12 2023-06-24T09:24:54.114 DEBUG:teuthology.parallel:result is None 2023-06-24T09:24:54.649 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : yajl-2.1.0-11.el8.x86_64 12/12 2023-06-24T09:24:54.649 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:24:54.649 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2023-06-24T09:24:54.650 INFO:teuthology.orchestra.run.smithi045.stdout: aardvark-dns-2:1.4.0-1.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:54.650 INFO:teuthology.orchestra.run.smithi045.stdout: buildah-1:1.28.2-2.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:54.650 INFO:teuthology.orchestra.run.smithi045.stdout: cockpit-podman-59-1.module_el8.8.0+1254+78119b6e.noarch 2023-06-24T09:24:54.650 INFO:teuthology.orchestra.run.smithi045.stdout: crun-1.7.2-1.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:54.650 INFO:teuthology.orchestra.run.smithi045.stdout: netavark-2:1.4.0-1.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:54.650 INFO:teuthology.orchestra.run.smithi045.stdout: python3-podman-4.3.0-2.module_el8.8.0+1254+78119b6e.noarch 2023-06-24T09:24:54.650 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-06-24T09:24:54.650 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-06-24T09:24:54.651 INFO:teuthology.orchestra.run.smithi045.stdout: skopeo-2:1.10.0-1.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:54.651 INFO:teuthology.orchestra.run.smithi045.stdout: toolbox-0.0.99.3-5.module_el8.8.0+1254+78119b6e.x86_64 2023-06-24T09:24:54.651 INFO:teuthology.orchestra.run.smithi045.stdout: udica-0.2.6-4.module_el8.8.0+1254+78119b6e.noarch 2023-06-24T09:24:54.651 INFO:teuthology.orchestra.run.smithi045.stdout: yajl-2.1.0-11.el8.x86_64 2023-06-24T09:24:54.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:24:54.651 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2023-06-24T09:24:54.987 DEBUG:teuthology.parallel:result is None 2023-06-24T09:24:54.987 INFO:teuthology.run_tasks:Running task nvme_loop... 2023-06-24T09:24:55.000 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2023-06-24T09:24:55.000 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:24:55.001 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2023-06-24T09:24:55.020 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-24T09:24:55.021 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2023-06-24T09:24:55.081 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-24T09:24:55.082 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-24T09:24:55.082 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 376419 Links: 1 2023-06-24T09:24:55.082 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-24T09:24:55.082 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-06-24T09:24:55.083 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-06-24 09:24:39.457240035 +0000 2023-06-24T09:24:55.083 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-06-24 09:24:11.594649255 +0000 2023-06-24T09:24:55.083 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-06-24 09:24:11.594649255 +0000 2023-06-24T09:24:55.083 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-06-24T09:24:55.084 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-24T09:24:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-06-24T09:24:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-06-24T09:24:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000104417 s, 4.9 MB/s 2023-06-24T09:24:55.150 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-24T09:24:55.210 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2023-06-24T09:24:55.269 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-24T09:24:55.270 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-24T09:24:55.270 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 374400 Links: 1 2023-06-24T09:24:55.270 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-24T09:24:55.270 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-06-24T09:24:55.270 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-06-24 09:24:39.457240035 +0000 2023-06-24T09:24:55.270 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-06-24 09:24:12.136641294 +0000 2023-06-24T09:24:55.270 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-06-24 09:24:12.136641294 +0000 2023-06-24T09:24:55.271 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-06-24T09:24:55.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-24T09:24:55.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-06-24T09:24:55.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-06-24T09:24:55.336 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000114328 s, 4.5 MB/s 2023-06-24T09:24:55.338 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-24T09:24:55.396 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2023-06-24T09:24:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-24T09:24:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-24T09:24:55.456 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 384205 Links: 1 2023-06-24T09:24:55.457 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-24T09:24:55.457 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-06-24T09:24:55.457 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-06-24 09:24:39.457240035 +0000 2023-06-24T09:24:55.457 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-06-24 09:24:12.674633393 +0000 2023-06-24T09:24:55.457 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-06-24 09:24:12.674633393 +0000 2023-06-24T09:24:55.457 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-06-24T09:24:55.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-24T09:24:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-06-24T09:24:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-06-24T09:24:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000119584 s, 4.3 MB/s 2023-06-24T09:24:55.525 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-24T09:24:55.585 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2023-06-24T09:24:55.644 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-24T09:24:55.644 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-24T09:24:55.644 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 381659 Links: 1 2023-06-24T09:24:55.645 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-24T09:24:55.645 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2023-06-24T09:24:55.645 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-06-24 09:24:39.457240035 +0000 2023-06-24T09:24:55.645 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-06-24 09:24:13.218625403 +0000 2023-06-24T09:24:55.645 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-06-24 09:24:13.218625403 +0000 2023-06-24T09:24:55.645 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-06-24T09:24:55.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-24T09:24:55.710 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-06-24T09:24:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-06-24T09:24:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.6824e-05 s, 5.3 MB/s 2023-06-24T09:24:55.712 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-24T09:24:55.771 DEBUG:teuthology.orchestra.run.smithi045:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2023-06-24T09:24:55.893 INFO:teuthology.orchestra.run.smithi045.stdout:loop 2023-06-24T09:24:55.894 INFO:tasks.nvme_loop:Connecting nvme_loop smithi045:/dev/vg_nvme/lv_1... 2023-06-24T09:24:55.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2023-06-24T09:24:55.931 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-06-24T09:24:55.966 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/vg_nvme/lv_11 2023-06-24T09:24:56.005 INFO:tasks.nvme_loop:Connecting nvme_loop smithi045:/dev/vg_nvme/lv_2... 2023-06-24T09:24:56.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2023-06-24T09:24:56.082 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-06-24T09:24:56.118 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/vg_nvme/lv_21 2023-06-24T09:24:56.156 INFO:tasks.nvme_loop:Connecting nvme_loop smithi045:/dev/vg_nvme/lv_3... 2023-06-24T09:24:56.157 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2023-06-24T09:24:56.233 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-06-24T09:24:56.268 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/vg_nvme/lv_31 2023-06-24T09:24:56.306 INFO:tasks.nvme_loop:Connecting nvme_loop smithi045:/dev/vg_nvme/lv_4... 2023-06-24T09:24:56.307 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2023-06-24T09:24:56.388 INFO:teuthology.orchestra.run.smithi045.stdout:1 2023-06-24T09:24:56.423 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/vg_nvme/lv_41 2023-06-24T09:24:56.460 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:24:56.461 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2023-06-24T09:24:56.518 DEBUG:teuthology.orchestra.run.smithi045:> sudo nvme list 2023-06-24T09:24:56.585 INFO:teuthology.orchestra.run.smithi045.stdout:Node SN Model Namespace Usage Format FW Rev 2023-06-24T09:24:56.585 INFO:teuthology.orchestra.run.smithi045.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2023-06-24T09:24:56.586 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/nvme0n1 PHMB75120098480DGN INTEL SSDPED1D480GA 1 480.10 GB / 480.10 GB 512 B + 0 B E2010325 2023-06-24T09:24:56.586 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/nvme1n1 2434940a0f36a8f91898 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 4.18.0-4 2023-06-24T09:24:56.586 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/nvme2n1 ea327aca602d1f674717 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 4.18.0-4 2023-06-24T09:24:56.586 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/nvme3n1 51d783ab236d0aa7745a Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 4.18.0-4 2023-06-24T09:24:56.586 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/nvme4n1 de742e8002e42c683f9d Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 4.18.0-4 2023-06-24T09:24:56.587 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2023-06-24T09:24:56.587 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:24:56.588 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/scratch_devs 2023-06-24T09:24:56.656 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-06-24T09:24:56.656 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2023-06-24T09:24:56.679 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-24T09:24:56.680 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2023-06-24T09:24:56.740 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-24T09:24:56.740 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-24T09:24:56.740 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 378457 Links: 1 2023-06-24T09:24:56.741 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-24T09:24:56.741 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-06-24T09:24:56.741 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-06-24 09:24:39.430188348 +0000 2023-06-24T09:24:56.741 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-06-24 09:24:01.707845211 +0000 2023-06-24T09:24:56.741 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-06-24 09:24:01.707845211 +0000 2023-06-24T09:24:56.742 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-06-24T09:24:56.742 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-24T09:24:56.809 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-06-24T09:24:56.810 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-06-24T09:24:56.810 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000268375 s, 1.9 MB/s 2023-06-24T09:24:56.811 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-24T09:24:56.872 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2023-06-24T09:24:56.931 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-24T09:24:56.932 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-24T09:24:56.932 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 373559 Links: 1 2023-06-24T09:24:56.932 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-24T09:24:56.932 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-06-24T09:24:56.932 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-06-24 09:24:39.430188348 +0000 2023-06-24T09:24:56.932 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-06-24 09:24:02.227836157 +0000 2023-06-24T09:24:56.933 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-06-24 09:24:02.227836157 +0000 2023-06-24T09:24:56.933 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-06-24T09:24:56.933 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-24T09:24:56.998 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-06-24T09:24:56.998 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-06-24T09:24:56.999 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000257898 s, 2.0 MB/s 2023-06-24T09:24:56.999 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-24T09:24:57.059 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2023-06-24T09:24:57.117 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-24T09:24:57.117 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-24T09:24:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 381354 Links: 1 2023-06-24T09:24:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-24T09:24:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-06-24T09:24:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-06-24 09:24:39.430188348 +0000 2023-06-24T09:24:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-06-24 09:24:02.767826754 +0000 2023-06-24T09:24:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-06-24 09:24:02.767826754 +0000 2023-06-24T09:24:57.118 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-06-24T09:24:57.119 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-24T09:24:57.184 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-06-24T09:24:57.184 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-06-24T09:24:57.184 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000225587 s, 2.3 MB/s 2023-06-24T09:24:57.185 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-24T09:24:57.244 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2023-06-24T09:24:57.302 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-24T09:24:57.303 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-24T09:24:57.303 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 381501 Links: 1 2023-06-24T09:24:57.303 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-24T09:24:57.303 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-06-24T09:24:57.303 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-06-24 09:24:39.430188348 +0000 2023-06-24T09:24:57.303 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-06-24 09:24:03.307817351 +0000 2023-06-24T09:24:57.303 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-06-24 09:24:03.307817351 +0000 2023-06-24T09:24:57.304 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-06-24T09:24:57.304 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-24T09:24:57.368 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-06-24T09:24:57.368 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-06-24T09:24:57.369 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000250875 s, 2.0 MB/s 2023-06-24T09:24:57.369 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-24T09:24:57.428 DEBUG:teuthology.orchestra.run.smithi188:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2023-06-24T09:24:57.543 INFO:teuthology.orchestra.run.smithi188.stdout:loop 2023-06-24T09:24:57.544 INFO:tasks.nvme_loop:Connecting nvme_loop smithi188:/dev/vg_nvme/lv_1... 2023-06-24T09:24:57.545 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2023-06-24T09:24:57.581 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-06-24T09:24:57.614 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/vg_nvme/lv_11 2023-06-24T09:24:57.653 INFO:tasks.nvme_loop:Connecting nvme_loop smithi188:/dev/vg_nvme/lv_2... 2023-06-24T09:24:57.654 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2023-06-24T09:24:57.729 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-06-24T09:24:57.762 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/vg_nvme/lv_21 2023-06-24T09:24:57.801 INFO:tasks.nvme_loop:Connecting nvme_loop smithi188:/dev/vg_nvme/lv_3... 2023-06-24T09:24:57.801 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2023-06-24T09:24:57.875 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-06-24T09:24:57.909 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/vg_nvme/lv_31 2023-06-24T09:24:57.946 INFO:tasks.nvme_loop:Connecting nvme_loop smithi188:/dev/vg_nvme/lv_4... 2023-06-24T09:24:57.947 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2023-06-24T09:24:58.022 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-06-24T09:24:58.055 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/vg_nvme/lv_41 2023-06-24T09:24:58.092 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-06-24T09:24:58.092 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2023-06-24T09:24:58.150 DEBUG:teuthology.orchestra.run.smithi188:> sudo nvme list 2023-06-24T09:24:58.216 INFO:teuthology.orchestra.run.smithi188.stdout:Node SN Model Namespace Usage Format FW Rev 2023-06-24T09:24:58.217 INFO:teuthology.orchestra.run.smithi188.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2023-06-24T09:24:58.217 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/nvme0n1 PHFT620500MP400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2023-06-24T09:24:58.217 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/nvme1n1 b42d0a8be2d82d4f8f50 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 4.18.0-4 2023-06-24T09:24:58.217 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/nvme2n1 4f5001aa5e24db854c1a Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 4.18.0-4 2023-06-24T09:24:58.217 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/nvme3n1 36cbbe884b0d8c522544 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 4.18.0-4 2023-06-24T09:24:58.217 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/nvme4n1 57528b3317ca68b9ec0c Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 4.18.0-4 2023-06-24T09:24:58.218 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2023-06-24T09:24:58.218 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-06-24T09:24:58.219 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/scratch_devs 2023-06-24T09:24:58.285 INFO:teuthology.run_tasks:Running task cephadm... 2023-06-24T09:24:58.384 INFO:tasks.cephadm:Config: {'roleless': True, 'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8'} 2023-06-24T09:24:58.385 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 2023-06-24T09:24:58.396 INFO:tasks.cephadm:Cluster fsid is fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:24:58.397 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-06-24T09:24:58.397 INFO:tasks.cephadm:No mon roles; fabricating mons 2023-06-24T09:24:58.398 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi045': '172.21.15.45', 'mon.smithi188': '172.21.15.188'} 2023-06-24T09:24:58.398 INFO:tasks.cephadm:Normalizing hostnames... 2023-06-24T09:24:58.399 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2023-06-24T09:24:58.434 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname $(hostname -s) 2023-06-24T09:24:58.461 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-06-24T09:24:58.463 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 2023-06-24T09:24:58.687 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/70785/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.1.0-166-gc9d917e3', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.1.0-166.gc9d917e3'}, 'url': 'https://1.chacra.ceph.com/r/ceph/reef-release/c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-06-23 17:05:00.580541', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef-release', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/reef-release/c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/52725/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.1.0-166-gc9d917e3', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.1.0-166.gc9d917e3'}, 'url': 'https://2.chacra.ceph.com/r/ceph/reef/c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-06-23 16:18:30.800481', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/reef/c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2023-06-24T09:24:58.828 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef-release, sha1 c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 2023-06-24T09:24:58.829 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef-release/c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8/centos/8/x86_64/flavors/default/cephadm 2023-06-24T09:24:58.829 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef-release/c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-06-24T09:24:59.220 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-rw-r--. 1 ubuntu ubuntu 648918 Jun 24 09:24 /home/ubuntu/cephtest/cephadm 2023-06-24T09:24:59.221 DEBUG:teuthology.orchestra.run.smithi188:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef-release/c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-06-24T09:24:59.617 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-rw-r--. 1 ubuntu ubuntu 648918 Jun 24 09:24 /home/ubuntu/cephtest/cephadm 2023-06-24T09:24:59.618 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-06-24T09:24:59.642 DEBUG:teuthology.orchestra.run.smithi188:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-06-24T09:24:59.672 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 on all hosts... 2023-06-24T09:24:59.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 pull 2023-06-24T09:24:59.683 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 pull 2023-06-24T09:25:01.441 INFO:teuthology.orchestra.run.smithi188.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8... 2023-06-24T09:25:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8... 2023-06-24T09:25:29.904 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2023-06-24T09:25:29.904 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 18.1.0-166-gc9d917e3 (c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8) reef (rc)", 2023-06-24T09:25:29.904 INFO:teuthology.orchestra.run.smithi188.stdout: "image_id": "8a246394989070c0a95f52c37d3ed7184850413eee93b84950fd63a703f5095a", 2023-06-24T09:25:29.904 INFO:teuthology.orchestra.run.smithi188.stdout: "repo_digests": [ 2023-06-24T09:25:29.905 INFO:teuthology.orchestra.run.smithi188.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4bb12992a5b8b78fe1e5b829f7a0d361efe5d8200d56fa3c47e437261d51929f" 2023-06-24T09:25:29.905 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2023-06-24T09:25:29.905 INFO:teuthology.orchestra.run.smithi188.stdout:} 2023-06-24T09:25:30.310 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2023-06-24T09:25:30.310 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 18.1.0-166-gc9d917e3 (c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8) reef (rc)", 2023-06-24T09:25:30.310 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "8a246394989070c0a95f52c37d3ed7184850413eee93b84950fd63a703f5095a", 2023-06-24T09:25:30.311 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2023-06-24T09:25:30.311 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4bb12992a5b8b78fe1e5b829f7a0d361efe5d8200d56fa3c47e437261d51929f" 2023-06-24T09:25:30.311 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2023-06-24T09:25:30.311 INFO:teuthology.orchestra.run.smithi045.stdout:} 2023-06-24T09:25:30.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2023-06-24T09:25:30.360 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph 2023-06-24T09:25:30.394 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2023-06-24T09:25:30.428 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /etc/ceph 2023-06-24T09:25:30.462 INFO:tasks.cephadm:Writing seed config... 2023-06-24T09:25:30.463 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-06-24T09:25:30.463 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-06-24T09:25:30.464 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-06-24T09:25:30.464 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-06-24T09:25:30.464 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-06-24T09:25:30.464 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-06-24T09:25:30.465 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-06-24T09:25:30.465 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-06-24T09:25:30.465 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:25:30.466 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-06-24T09:25:30.485 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = fceaf654-1270-11ee-9b2b-001a4aab830c [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2023-06-24T09:25:30.486 DEBUG:teuthology.orchestra.run.smithi045:mon.smithi045> sudo journalctl -f -n 0 -u ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mon.smithi045.service 2023-06-24T09:25:30.527 INFO:tasks.cephadm:Bootstrapping... 2023-06-24T09:25:30.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 -v bootstrap --fsid fceaf654-1270-11ee-9b2b-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.45 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-06-24T09:25:30.555 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:-- Logs begin at Sat 2023-06-24 09:16:53 UTC. -- 2023-06-24T09:25:30.693 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2023-06-24T09:25:30.694 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8', '-v', 'bootstrap', '--fsid', 'fceaf654-1270-11ee-9b2b-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.45', '--skip-admin-label'] 2023-06-24T09:25:30.733 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.3.1 2023-06-24T09:25:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-06-24T09:25:30.734 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2023-06-24T09:25:30.771 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.3.1 2023-06-24T09:25:30.771 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2023-06-24T09:25:30.771 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2023-06-24T09:25:30.778 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-06-24T09:25:30.778 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-06-24T09:25:30.784 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-06-24T09:25:30.785 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-06-24T09:25:30.791 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2023-06-24T09:25:30.796 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2023-06-24T09:25:30.797 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2023-06-24T09:25:30.797 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2023-06-24T09:25:30.834 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.3.1 2023-06-24T09:25:30.835 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 4.3.1 is present 2023-06-24T09:25:30.835 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2023-06-24T09:25:30.835 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2023-06-24T09:25:30.841 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-06-24T09:25:30.841 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-06-24T09:25:30.846 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-06-24T09:25:30.847 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-06-24T09:25:30.852 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2023-06-24T09:25:30.858 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2023-06-24T09:25:30.858 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2023-06-24T09:25:30.859 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2023-06-24T09:25:30.859 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:25:30.859 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 139687650110320 on /run/cephadm/fceaf654-1270-11ee-9b2b-001a4aab830c.lock 2023-06-24T09:25:30.859 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 139687650110320 acquired on /run/cephadm/fceaf654-1270-11ee-9b2b-001a4aab830c.lock 2023-06-24T09:25:30.860 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 3300 ... 2023-06-24T09:25:30.860 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2023-06-24T09:25:30.861 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:3300, 172.21.15.45:6789], mon addrv is [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2023-06-24T09:25:30.863 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.45 metric 100 2023-06-24T09:25:30.864 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2023-06-24T09:25:30.866 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-06-24T09:25:30.867 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-06-24T09:25:30.867 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1791sec hoplimit 64 pref medium 2023-06-24T09:25:30.869 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-06-24T09:25:30.869 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-06-24T09:25:30.869 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-06-24T09:25:30.869 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-06-24T09:25:30.869 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link 2023-06-24T09:25:30.870 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-06-24T09:25:30.870 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2023-06-24T09:25:30.870 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2023-06-24T09:25:30.870 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-06-24T09:25:30.871 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-06-24T09:25:30.871 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8... 2023-06-24T09:25:31.274 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 8a246394989070c0a95f52c37d3ed7184850413eee93b84950fd63a703f5095a 2023-06-24T09:25:31.274 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8... 2023-06-24T09:25:31.274 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2023-06-24T09:25:31.275 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:0146dfe6837e9a15f20c43b946697b8dd2baff474fcb43ad34c16f2a81a9e92f 2023-06-24T09:25:31.275 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:2190a7c58a810ef42ff112928214340ca1d477693415e4eeb7f8e46eb1ea0ced 2023-06-24T09:25:31.275 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:8a246394989070c0a95f52c37d3ed7184850413eee93b84950fd63a703f5095a 2023-06-24T09:25:31.275 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2023-06-24T09:25:31.275 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Storing signatures 2023-06-24T09:25:31.996 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 18.1.0-166-gc9d917e3 (c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8) reef (rc) 2023-06-24T09:25:31.996 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 18.1.0-166-gc9d917e3 (c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8) reef (rc) 2023-06-24T09:25:31.996 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2023-06-24T09:25:32.624 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2023-06-24T09:25:32.625 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2023-06-24T09:25:33.282 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCMtpZku0oJNxAA882x1Xu0v/sNV2b+wDw7PQ== 2023-06-24T09:25:33.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCNtpZkhuMyIRAAfEemxRin7Ftn+05dfJDsfQ== 2023-06-24T09:25:34.517 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCOtpZkEHqbChAAfR3vJ5rGBx2a1GrMUSt43A== 2023-06-24T09:25:34.518 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2023-06-24T09:25:35.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-06-24T09:25:35.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-06-24T09:25:35.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:25:35.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-06-24T09:25:35.196 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for smithi045 [v2:172.21.15.45:3300,v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-06-24T09:25:35.196 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = pacific 2023-06-24T09:25:35.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:25:35.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-06-24T09:25:35.197 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:25:35.197 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2023-06-24T09:25:35.886 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.smithi045 on 2023-06-24T09:25:36.088 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-06-24T09:25:36.189 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-fceaf654-1270-11ee-9b2b-001a4aab830c.target → /etc/systemd/system/ceph-fceaf654-1270-11ee-9b2b-001a4aab830c.target. 2023-06-24T09:25:36.189 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-fceaf654-1270-11ee-9b2b-001a4aab830c.target → /etc/systemd/system/ceph-fceaf654-1270-11ee-9b2b-001a4aab830c.target. 2023-06-24T09:25:36.331 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mon.smithi045 2023-06-24T09:25:36.332 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mon.smithi045.service: Unit ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mon.smithi045.service not loaded. 2023-06-24T09:25:36.434 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fceaf654-1270-11ee-9b2b-001a4aab830c.target.wants/ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mon.smithi045.service → /etc/systemd/system/ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@.service. 2023-06-24T09:25:36.913 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-24T09:25:36.913 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-06-24T09:25:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-24T09:25:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-06-24T09:25:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-06-24T09:25:36.921 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2023-06-24T09:25:36.922 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2023-06-24T09:25:36.922 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2023-06-24T09:25:37.149 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:36 smithi045 ceph-mon[102107]: mon.smithi045 is new leader, mons smithi045 in quorum (ranks 0) 2023-06-24T09:25:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2023-06-24T09:25:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:25:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-06-24T09:25:38.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2023-06-24T09:25:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi045 (age 0.495083s) 2023-06-24T09:25:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-06-24T09:25:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-06-24T09:25:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2023-06-24T09:25:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-06-24T09:25:38.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-06-24T09:25:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-06-24T09:25:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2023-06-24T09:25:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2023-06-24T09:25:38.226 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2023-06-24T09:25:38.227 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:37 smithi045 ceph-mon[102107]: mon.smithi045 is new leader, mons smithi045 in quorum (ranks 0) 2023-06-24T09:25:38.227 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:37 smithi045 ceph-mon[102107]: monmap e1: 1 mons at {smithi045=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0]} removed_ranks: {} 2023-06-24T09:25:38.227 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:37 smithi045 ceph-mon[102107]: fsmap 2023-06-24T09:25:38.227 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:37 smithi045 ceph-mon[102107]: osdmap e1: 0 total, 0 up, 0 in 2023-06-24T09:25:38.227 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:37 smithi045 ceph-mon[102107]: mgrmap e1: no daemons active 2023-06-24T09:25:38.227 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:37 smithi045 ceph-mon[102107]: from='client.? 172.21.15.45:0/2522298584' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-06-24T09:25:39.058 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:38 smithi045 ceph-mon[102107]: from='client.? 172.21.15.45:0/2972882375' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-06-24T09:25:39.058 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:38 smithi045 ceph-mon[102107]: from='client.? 172.21.15.45:0/2972882375' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-06-24T09:25:39.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2023-06-24T09:25:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:25:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2023-06-24T09:25:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-06-24T09:25:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-06-24T09:25:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-06-24T09:25:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-06-24T09:25:39.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2023-06-24T09:25:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-06-24T09:25:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2023-06-24T09:25:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-06-24T09:25:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-06-24T09:25:39.084 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2023-06-24T09:25:39.889 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2023-06-24T09:25:40.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:39 smithi045 systemd[1]: Stopping Ceph mon.smithi045 for fceaf654-1270-11ee-9b2b-001a4aab830c... 2023-06-24T09:25:40.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:39 smithi045 ceph-mon[102107]: from='client.? 172.21.15.45:0/2145636723' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:25:40.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:40 smithi045 ceph-fceaf654-1270-11ee-9b2b-001a4aab830c-mon-smithi045[102103]: 2023-06-24T09:25:40.080+0000 7f3885b13700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi045 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-24T09:25:40.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:40 smithi045 ceph-fceaf654-1270-11ee-9b2b-001a4aab830c-mon-smithi045[102103]: 2023-06-24T09:25:40.080+0000 7f3885b13700 -1 mon.smithi045@0(leader) e1 *** Got Signal Terminated *** 2023-06-24T09:25:40.571 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:40 smithi045 bash[102369]: ceph-fceaf654-1270-11ee-9b2b-001a4aab830c-mon-smithi045 2023-06-24T09:25:40.867 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:40 smithi045 systemd[1]: ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mon.smithi045.service: Succeeded. 2023-06-24T09:25:40.867 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:40 smithi045 systemd[1]: Stopped Ceph mon.smithi045 for fceaf654-1270-11ee-9b2b-001a4aab830c. 2023-06-24T09:25:40.868 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:40 smithi045 systemd[1]: Starting Ceph mon.smithi045 for fceaf654-1270-11ee-9b2b-001a4aab830c... 2023-06-24T09:25:41.029 INFO:teuthology.orchestra.run.smithi045.stdout:Setting mon public_network to 172.21.0.0/20 2023-06-24T09:25:41.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:40 smithi045 podman[102489]: 2023-06-24T09:25:41.362 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: set uid:gid to 167:167 (ceph:ceph) 2023-06-24T09:25:41.363 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: ceph version 18.1.0-166-gc9d917e3 (c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8) reef (rc), process ceph-mon, pid 2 2023-06-24T09:25:41.363 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: pidfile_write: ignore empty --pid-file 2023-06-24T09:25:41.363 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: load: jerasure load: lrc 2023-06-24T09:25:41.364 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: RocksDB version: 7.9.2 2023-06-24T09:25:41.364 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Git sha 0 2023-06-24T09:25:41.364 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Compile date 2023-06-23 16:06:32 2023-06-24T09:25:41.364 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: DB SUMMARY 2023-06-24T09:25:41.364 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: DB Session ID: E8PQC9AWFJUE22KWW6YZ 2023-06-24T09:25:41.365 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: CURRENT file: CURRENT 2023-06-24T09:25:41.365 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: IDENTITY file: IDENTITY 2023-06-24T09:25:41.365 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-06-24T09:25:41.365 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi045/store.db dir, Total Num: 1, files: 000008.sst 2023-06-24T09:25:41.365 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi045/store.db: 000009.log size: 83670 ; 2023-06-24T09:25:41.365 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.error_if_exists: 0 2023-06-24T09:25:41.365 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.create_if_missing: 0 2023-06-24T09:25:41.366 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.paranoid_checks: 1 2023-06-24T09:25:41.366 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.flush_verify_memtable_count: 1 2023-06-24T09:25:41.366 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-06-24T09:25:41.366 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-06-24T09:25:41.366 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.env: 0x55da1b94a720 2023-06-24T09:25:41.366 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.fs: PosixFileSystem 2023-06-24T09:25:41.366 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.info_log: 0x55da1d6852a0 2023-06-24T09:25:41.366 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_file_opening_threads: 16 2023-06-24T09:25:41.367 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.statistics: (nil) 2023-06-24T09:25:41.367 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.use_fsync: 0 2023-06-24T09:25:41.367 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_log_file_size: 0 2023-06-24T09:25:41.367 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-06-24T09:25:41.367 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.log_file_time_to_roll: 0 2023-06-24T09:25:41.367 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.keep_log_file_num: 1000 2023-06-24T09:25:41.367 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.recycle_log_file_num: 0 2023-06-24T09:25:41.367 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.allow_fallocate: 1 2023-06-24T09:25:41.368 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.allow_mmap_reads: 0 2023-06-24T09:25:41.368 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.allow_mmap_writes: 0 2023-06-24T09:25:41.368 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.use_direct_reads: 0 2023-06-24T09:25:41.368 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-06-24T09:25:41.368 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.create_missing_column_families: 0 2023-06-24T09:25:41.368 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.db_log_dir: 2023-06-24T09:25:41.368 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.wal_dir: 2023-06-24T09:25:41.369 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.table_cache_numshardbits: 6 2023-06-24T09:25:41.369 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.WAL_ttl_seconds: 0 2023-06-24T09:25:41.369 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.WAL_size_limit_MB: 0 2023-06-24T09:25:41.369 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-06-24T09:25:41.369 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-06-24T09:25:41.369 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.is_fd_close_on_exec: 1 2023-06-24T09:25:41.369 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.advise_random_on_open: 1 2023-06-24T09:25:41.369 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.db_write_buffer_size: 0 2023-06-24T09:25:41.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.write_buffer_manager: 0x55da1c9165a0 2023-06-24T09:25:41.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-06-24T09:25:41.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-06-24T09:25:41.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.use_adaptive_mutex: 0 2023-06-24T09:25:41.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.rate_limiter: (nil) 2023-06-24T09:25:41.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-06-24T09:25:41.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.wal_recovery_mode: 2 2023-06-24T09:25:41.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.enable_thread_tracking: 0 2023-06-24T09:25:41.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.enable_pipelined_write: 0 2023-06-24T09:25:41.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.unordered_write: 0 2023-06-24T09:25:41.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-06-24T09:25:41.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-06-24T09:25:41.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-06-24T09:25:41.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-06-24T09:25:41.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.row_cache: None 2023-06-24T09:25:41.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.wal_filter: None 2023-06-24T09:25:41.372 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-06-24T09:25:41.372 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.allow_ingest_behind: 0 2023-06-24T09:25:41.372 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.two_write_queues: 0 2023-06-24T09:25:41.372 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.manual_wal_flush: 0 2023-06-24T09:25:41.372 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.wal_compression: 0 2023-06-24T09:25:41.372 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.atomic_flush: 0 2023-06-24T09:25:41.372 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-06-24T09:25:41.372 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.persist_stats_to_disk: 0 2023-06-24T09:25:41.373 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.write_dbid_to_manifest: 0 2023-06-24T09:25:41.373 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.log_readahead_size: 0 2023-06-24T09:25:41.373 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-06-24T09:25:41.373 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.best_efforts_recovery: 0 2023-06-24T09:25:41.373 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-06-24T09:25:41.373 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-06-24T09:25:41.373 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.allow_data_in_errors: 0 2023-06-24T09:25:41.374 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.db_host_id: __hostname__ 2023-06-24T09:25:41.375 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.enforce_single_del_contracts: true 2023-06-24T09:25:41.375 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_background_jobs: 2 2023-06-24T09:25:41.375 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_background_compactions: -1 2023-06-24T09:25:41.375 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_subcompactions: 1 2023-06-24T09:25:41.375 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-06-24T09:25:41.375 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-06-24T09:25:41.375 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.delayed_write_rate : 16777216 2023-06-24T09:25:41.376 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_total_wal_size: 0 2023-06-24T09:25:41.376 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-06-24T09:25:41.376 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.stats_dump_period_sec: 600 2023-06-24T09:25:41.376 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.stats_persist_period_sec: 600 2023-06-24T09:25:41.376 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-06-24T09:25:41.376 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_open_files: -1 2023-06-24T09:25:41.376 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bytes_per_sync: 0 2023-06-24T09:25:41.377 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.wal_bytes_per_sync: 0 2023-06-24T09:25:41.377 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.strict_bytes_per_sync: 0 2023-06-24T09:25:41.377 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_readahead_size: 0 2023-06-24T09:25:41.377 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_background_flushes: -1 2023-06-24T09:25:41.377 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Compression algorithms supported: 2023-06-24T09:25:41.377 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-06-24T09:25:41.377 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: kZSTD supported: 0 2023-06-24T09:25:41.378 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: kXpressCompression supported: 0 2023-06-24T09:25:41.378 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: kLZ4HCCompression supported: 1 2023-06-24T09:25:41.378 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: kZlibCompression supported: 1 2023-06-24T09:25:41.378 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: kSnappyCompression supported: 1 2023-06-24T09:25:41.378 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: kLZ4Compression supported: 1 2023-06-24T09:25:41.378 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: kBZip2Compression supported: 0 2023-06-24T09:25:41.378 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Fast CRC32 supported: Supported on x86 2023-06-24T09:25:41.379 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: DMutex implementation: pthread_mutex_t 2023-06-24T09:25:41.379 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi045/store.db/MANIFEST-000010 2023-06-24T09:25:41.379 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-06-24T09:25:41.379 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-06-24T09:25:41.379 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.merge_operator: 2023-06-24T09:25:41.379 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_filter: None 2023-06-24T09:25:41.379 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_filter_factory: None 2023-06-24T09:25:41.380 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.sst_partitioner_factory: None 2023-06-24T09:25:41.380 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.memtable_factory: SkipListFactory 2023-06-24T09:25:41.380 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.table_factory: BlockBasedTable 2023-06-24T09:25:41.380 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55da1d6853a0) 2023-06-24T09:25:41.380 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: cache_index_and_filter_blocks: 1 2023-06-24T09:25:41.380 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-06-24T09:25:41.380 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-06-24T09:25:41.380 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: pin_top_level_index_and_filter: 1 2023-06-24T09:25:41.381 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: index_type: 0 2023-06-24T09:25:41.381 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: data_block_index_type: 0 2023-06-24T09:25:41.381 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: index_shortening: 1 2023-06-24T09:25:41.381 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2023-06-24T09:25:41.381 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: checksum: 4 2023-06-24T09:25:41.381 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: no_block_cache: 0 2023-06-24T09:25:41.381 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: block_cache: 0x55da1c9971f0 2023-06-24T09:25:41.381 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: block_cache_name: BinnedLRUCache 2023-06-24T09:25:41.382 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: block_cache_options: 2023-06-24T09:25:41.382 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: capacity : 536870912 2023-06-24T09:25:41.382 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: num_shard_bits : 4 2023-06-24T09:25:41.382 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: strict_capacity_limit : 0 2023-06-24T09:25:41.382 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: high_pri_pool_ratio: 0.000 2023-06-24T09:25:41.382 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: block_cache_compressed: (nil) 2023-06-24T09:25:41.382 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: persistent_cache: (nil) 2023-06-24T09:25:41.382 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: block_size: 4096 2023-06-24T09:25:41.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: block_size_deviation: 10 2023-06-24T09:25:41.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: block_restart_interval: 16 2023-06-24T09:25:41.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: index_block_restart_interval: 1 2023-06-24T09:25:41.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: metadata_block_size: 4096 2023-06-24T09:25:41.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: partition_filters: 0 2023-06-24T09:25:41.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: use_delta_encoding: 1 2023-06-24T09:25:41.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: filter_policy: bloomfilter 2023-06-24T09:25:41.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: whole_key_filtering: 1 2023-06-24T09:25:41.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: verify_compression: 0 2023-06-24T09:25:41.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: read_amp_bytes_per_bit: 0 2023-06-24T09:25:41.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: format_version: 5 2023-06-24T09:25:41.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: enable_index_compression: 1 2023-06-24T09:25:41.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: block_align: 0 2023-06-24T09:25:41.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: max_auto_readahead_size: 262144 2023-06-24T09:25:41.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: prepopulate_block_cache: 0 2023-06-24T09:25:41.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: initial_auto_readahead_size: 8192 2023-06-24T09:25:41.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2023-06-24T09:25:41.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.write_buffer_size: 33554432 2023-06-24T09:25:41.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_write_buffer_number: 2 2023-06-24T09:25:41.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compression: NoCompression 2023-06-24T09:25:41.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bottommost_compression: Disabled 2023-06-24T09:25:41.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.prefix_extractor: nullptr 2023-06-24T09:25:41.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-06-24T09:25:41.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.num_levels: 7 2023-06-24T09:25:41.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-06-24T09:25:41.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-06-24T09:25:41.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-06-24T09:25:41.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-06-24T09:25:41.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-06-24T09:25:41.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-06-24T09:25:41.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-06-24T09:25:41.387 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-06-24T09:25:41.387 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-06-24T09:25:41.387 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-06-24T09:25:41.387 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-06-24T09:25:41.387 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-06-24T09:25:41.387 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compression_opts.window_bits: -14 2023-06-24T09:25:41.387 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compression_opts.level: 32767 2023-06-24T09:25:41.387 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compression_opts.strategy: 0 2023-06-24T09:25:41.388 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-06-24T09:25:41.388 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-06-24T09:25:41.388 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-06-24T09:25:41.388 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-06-24T09:25:41.388 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compression_opts.enabled: false 2023-06-24T09:25:41.388 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-06-24T09:25:41.388 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-06-24T09:25:41.388 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-06-24T09:25:41.389 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-06-24T09:25:41.389 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.target_file_size_base: 67108864 2023-06-24T09:25:41.389 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.target_file_size_multiplier: 1 2023-06-24T09:25:41.389 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-06-24T09:25:41.389 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-06-24T09:25:41.389 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-06-24T09:25:41.389 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-06-24T09:25:41.389 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-06-24T09:25:41.390 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-06-24T09:25:41.390 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-06-24T09:25:41.390 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-06-24T09:25:41.390 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-06-24T09:25:41.390 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-06-24T09:25:41.390 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-06-24T09:25:41.390 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-06-24T09:25:41.390 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-06-24T09:25:41.391 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.arena_block_size: 1048576 2023-06-24T09:25:41.391 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-06-24T09:25:41.391 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-06-24T09:25:41.391 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.disable_auto_compactions: 0 2023-06-24T09:25:41.391 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-06-24T09:25:41.391 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-06-24T09:25:41.391 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-06-24T09:25:41.391 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-06-24T09:25:41.392 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-06-24T09:25:41.392 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-06-24T09:25:41.392 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-06-24T09:25:41.392 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-06-24T09:25:41.392 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-06-24T09:25:41.392 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-06-24T09:25:41.392 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.table_properties_collectors: 2023-06-24T09:25:41.392 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.inplace_update_support: 0 2023-06-24T09:25:41.393 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.inplace_update_num_locks: 10000 2023-06-24T09:25:41.393 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-06-24T09:25:41.393 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-06-24T09:25:41.393 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.memtable_huge_page_size: 0 2023-06-24T09:25:41.393 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.bloom_locality: 0 2023-06-24T09:25:41.393 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.max_successive_merges: 0 2023-06-24T09:25:41.393 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.optimize_filters_for_hits: 0 2023-06-24T09:25:41.393 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.paranoid_file_checks: 0 2023-06-24T09:25:41.394 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.force_consistency_checks: 1 2023-06-24T09:25:41.394 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.report_bg_io_stats: 0 2023-06-24T09:25:41.394 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.ttl: 2592000 2023-06-24T09:25:41.394 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.periodic_compaction_seconds: 0 2023-06-24T09:25:41.394 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-06-24T09:25:41.394 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-06-24T09:25:41.394 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.enable_blob_files: false 2023-06-24T09:25:41.394 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.min_blob_size: 0 2023-06-24T09:25:41.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.blob_file_size: 268435456 2023-06-24T09:25:41.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.blob_compression_type: NoCompression 2023-06-24T09:25:41.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.enable_blob_garbage_collection: false 2023-06-24T09:25:41.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-06-24T09:25:41.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-06-24T09:25:41.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-06-24T09:25:41.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.blob_file_starting_level: 0 2023-06-24T09:25:41.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-06-24T09:25:41.396 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi045/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-06-24T09:25:41.396 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-06-24T09:25:41.396 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 24aa5e49-6800-40fc-94bc-0736767c644c 2023-06-24T09:25:41.396 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687598741007071, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-06-24T09:25:41.396 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-06-24T09:25:41.397 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687598741008397, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 80442, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 248, "table_properties": {"data_size": 78625, "index_size": 227, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9753, "raw_average_key_size": 47, "raw_value_size": 73045, "raw_average_value_size": 352, "num_data_blocks": 10, "num_entries": 207, "num_filter_entries": 207, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1687598741, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "24aa5e49-6800-40fc-94bc-0736767c644c", "db_session_id": "E8PQC9AWFJUE22KWW6YZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-06-24T09:25:41.397 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687598741008495, "job": 1, "event": "recovery_finished"} 2023-06-24T09:25:41.397 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-06-24T09:25:41.397 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi045/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-06-24T09:25:41.397 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55da1ca34000 2023-06-24T09:25:41.397 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: DB pointer 0x55da1ca20000 2023-06-24T09:25:41.397 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-06-24T09:25:41.398 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-06-24T09:25:41.398 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: ** DB Stats ** 2023-06-24T09:25:41.398 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-24T09:25:41.398 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-06-24T09:25:41.398 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-24T09:25:41.398 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-24T09:25:41.398 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-06-24T09:25:41.398 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-24T09:25:41.399 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-24T09:25:41.399 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: 2023-06-24T09:25:41.399 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: ** Compaction Stats [default] ** 2023-06-24T09:25:41.399 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-06-24T09:25:41.399 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-06-24T09:25:41.399 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: L0 2/0 80.43 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-24T09:25:41.399 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Sum 2/0 80.43 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-24T09:25:41.399 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-24T09:25:41.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: 2023-06-24T09:25:41.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: ** Compaction Stats [default] ** 2023-06-24T09:25:41.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-06-24T09:25:41.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-24T09:25:41.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-24T09:25:41.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: 2023-06-24T09:25:41.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-06-24T09:25:41.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: 2023-06-24T09:25:41.401 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-24T09:25:41.401 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-24T09:25:41.401 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-24T09:25:41.401 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-24T09:25:41.401 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-24T09:25:41.401 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-24T09:25:41.401 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Cumulative compaction: 0.00 GB write, 17.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-24T09:25:41.401 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Interval compaction: 0.00 GB write, 17.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-24T09:25:41.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-24T09:25:41.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Block cache BinnedLRUCache@0x55da1c9971f0#2 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-06-24T09:25:41.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2023-06-24T09:25:41.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: 2023-06-24T09:25:41.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-24T09:25:41.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: starting mon.smithi045 rank 0 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi045 fsid fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:25:41.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: mon.smithi045@-1(???) e1 preinit fsid fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:25:41.403 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: mon.smithi045@-1(???).mds e1 new map 2023-06-24T09:25:41.403 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: mon.smithi045@-1(???).mds e1 print_map 2023-06-24T09:25:41.403 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: e1 2023-06-24T09:25:41.403 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-06-24T09:25:41.403 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-06-24T09:25:41.403 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: legacy client fscid: -1 2023-06-24T09:25:41.403 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: 2023-06-24T09:25:41.403 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout: No filesystems configured 2023-06-24T09:25:41.404 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: mon.smithi045@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-06-24T09:25:41.404 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: mon.smithi045@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-24T09:25:41.404 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: mon.smithi045@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-24T09:25:41.404 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: mon.smithi045@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-24T09:25:41.404 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: mon.smithi045@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-06-24T09:25:41.404 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta expand map: {default=false} 2023-06-24T09:25:41.404 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta from 'false' to 'false' 2023-06-24T09:25:41.404 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta expanded map: {default=false} 2023-06-24T09:25:41.405 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta expand map: {default=info} 2023-06-24T09:25:41.405 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta from 'info' to 'info' 2023-06-24T09:25:41.405 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta expanded map: {default=info} 2023-06-24T09:25:41.405 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta expand map: {default=daemon} 2023-06-24T09:25:41.405 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta from 'daemon' to 'daemon' 2023-06-24T09:25:41.405 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta expanded map: {default=daemon} 2023-06-24T09:25:41.405 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta expand map: {default=debug} 2023-06-24T09:25:41.405 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta from 'debug' to 'debug' 2023-06-24T09:25:41.406 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: expand_channel_meta expanded map: {default=debug} 2023-06-24T09:25:41.406 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 bash[102489]: a4e89356a2e12727d804e5cbef6a6f196650ebbf892e5fc4caca6088f17538c6 2023-06-24T09:25:41.406 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 systemd[1]: Started Ceph mon.smithi045 for fceaf654-1270-11ee-9b2b-001a4aab830c. 2023-06-24T09:25:41.406 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: mon.smithi045 is new leader, mons smithi045 in quorum (ranks 0) 2023-06-24T09:25:41.406 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: monmap e1: 1 mons at {smithi045=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0]} removed_ranks: {} 2023-06-24T09:25:41.406 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: fsmap 2023-06-24T09:25:41.406 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: osdmap e1: 0 total, 0 up, 0 in 2023-06-24T09:25:41.406 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:41 smithi045 ceph-mon[102509]: mgrmap e1: no daemons active 2023-06-24T09:25:41.890 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2023-06-24T09:25:41.890 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-06-24T09:25:41.890 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2023-06-24T09:25:41.891 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 9283 ... 2023-06-24T09:25:41.891 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 8765 ... 2023-06-24T09:25:41.891 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 8443 ... 2023-06-24T09:25:42.035 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mgr.smithi045.dcgntd 2023-06-24T09:25:42.036 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mgr.smithi045.dcgntd.service: Unit ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mgr.smithi045.dcgntd.service not loaded. 2023-06-24T09:25:42.132 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fceaf654-1270-11ee-9b2b-001a4aab830c.target.wants/ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mgr.smithi045.dcgntd.service → /etc/systemd/system/ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@.service. 2023-06-24T09:25:42.617 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-24T09:25:42.618 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-06-24T09:25:42.626 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-24T09:25:42.626 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-06-24T09:25:42.626 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-06-24T09:25:42.626 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2023-06-24T09:25:42.634 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-24T09:25:42.634 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-06-24T09:25:42.643 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-24T09:25:42.643 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-06-24T09:25:42.643 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-06-24T09:25:42.643 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2023-06-24T09:25:42.643 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2023-06-24T09:25:42.644 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2023-06-24T09:25:42.744 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:42 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3567006040' entity='client.admin' 2023-06-24T09:25:43.810 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:43 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/4283130493' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-24T09:25:44.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:44.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-06-24T09:25:44.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "fceaf654-1270-11ee-9b2b-001a4aab830c", 2023-06-24T09:25:44.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2023-06-24T09:25:44.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-24T09:25:44.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-24T09:25:44.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-24T09:25:44.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:44.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-24T09:25:44.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-24T09:25:44.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2023-06-24T09:25:44.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:44.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-24T09:25:44.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "smithi045" 2023-06-24T09:25:44.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:44.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-06-24T09:25:44.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-24T09:25:44.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:44.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-06-24T09:25:44.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-24T09:25:44.901 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:44.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-24T09:25:44.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:44.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-24T09:25:44.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-24T09:25:44.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-24T09:25:44.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-24T09:25:44.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-24T09:25:44.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-24T09:25:44.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:44.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-24T09:25:44.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-24T09:25:44.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-24T09:25:44.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-24T09:25:44.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-24T09:25:44.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-24T09:25:44.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-24T09:25:44.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-24T09:25:44.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-24T09:25:44.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:44.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-24T09:25:44.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:44.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-24T09:25:44.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-24T09:25:44.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:44.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-24T09:25:44.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2023-06-24T09:25:44.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-24T09:25:44.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-24T09:25:44.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-24T09:25:44.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-24T09:25:44.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2023-06-24T09:25:44.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:44.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-24T09:25:44.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:44.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-24T09:25:44.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:44.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2023-06-24T09:25:36.937607+0000", 2023-06-24T09:25:44.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-24T09:25:44.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:44.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-24T09:25:44.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-06-24T09:25:44.910 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2023-06-24T09:25:47.810 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:47.810 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-06-24T09:25:47.810 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "fceaf654-1270-11ee-9b2b-001a4aab830c", 2023-06-24T09:25:47.811 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2023-06-24T09:25:47.811 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-24T09:25:47.811 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-24T09:25:47.811 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-24T09:25:47.812 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:47.812 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-24T09:25:47.812 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-24T09:25:47.813 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2023-06-24T09:25:47.813 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:47.813 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-24T09:25:47.813 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "smithi045" 2023-06-24T09:25:47.814 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:47.814 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-06-24T09:25:47.814 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-24T09:25:47.814 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:47.814 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-06-24T09:25:47.814 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-24T09:25:47.814 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:47.815 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-24T09:25:47.815 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:47.815 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-24T09:25:47.815 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-24T09:25:47.816 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-24T09:25:47.816 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-24T09:25:47.816 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-24T09:25:47.816 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-24T09:25:47.816 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:47.817 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-24T09:25:47.817 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-24T09:25:47.817 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-24T09:25:47.817 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-24T09:25:47.817 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-24T09:25:47.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-24T09:25:47.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-24T09:25:47.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-24T09:25:47.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-24T09:25:47.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:47.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-24T09:25:47.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:47.818 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-24T09:25:47.819 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-24T09:25:47.819 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:47.819 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-24T09:25:47.819 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2023-06-24T09:25:47.819 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-24T09:25:47.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-24T09:25:47.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-24T09:25:47.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-24T09:25:47.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2023-06-24T09:25:47.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:47.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-24T09:25:47.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:47.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-24T09:25:47.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:47.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2023-06-24T09:25:36.937607+0000", 2023-06-24T09:25:47.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-24T09:25:47.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:47.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-24T09:25:47.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-06-24T09:25:47.822 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2023-06-24T09:25:47.822 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:47 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3819972769' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-24T09:25:49.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: Activating manager daemon smithi045.dcgntd 2023-06-24T09:25:49.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: mgrmap e2: smithi045.dcgntd(active, starting, since 0.00339629s) 2023-06-24T09:25:49.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: from='mgr.14100 172.21.15.45:0/8310826' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-24T09:25:49.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: from='mgr.14100 172.21.15.45:0/8310826' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-24T09:25:49.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: from='mgr.14100 172.21.15.45:0/8310826' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-24T09:25:49.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: from='mgr.14100 172.21.15.45:0/8310826' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi045"}]: dispatch 2023-06-24T09:25:49.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: from='mgr.14100 172.21.15.45:0/8310826' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr metadata", "who": "smithi045.dcgntd", "id": "smithi045.dcgntd"}]: dispatch 2023-06-24T09:25:49.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: Manager daemon smithi045.dcgntd is now available 2023-06-24T09:25:49.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: from='mgr.14100 172.21.15.45:0/8310826' entity='mgr.smithi045.dcgntd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi045.dcgntd/mirror_snapshot_schedule"}]: dispatch 2023-06-24T09:25:49.673 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: from='mgr.14100 172.21.15.45:0/8310826' entity='mgr.smithi045.dcgntd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi045.dcgntd/trash_purge_schedule"}]: dispatch 2023-06-24T09:25:49.673 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: from='mgr.14100 172.21.15.45:0/8310826' entity='mgr.smithi045.dcgntd' 2023-06-24T09:25:49.673 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: from='mgr.14100 172.21.15.45:0/8310826' entity='mgr.smithi045.dcgntd' 2023-06-24T09:25:49.673 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:49 smithi045 ceph-mon[102509]: from='mgr.14100 172.21.15.45:0/8310826' entity='mgr.smithi045.dcgntd' 2023-06-24T09:25:50.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:50.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-06-24T09:25:50.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "fceaf654-1270-11ee-9b2b-001a4aab830c", 2023-06-24T09:25:50.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2023-06-24T09:25:50.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-24T09:25:50.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-24T09:25:50.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-24T09:25:50.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:50.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-24T09:25:50.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-24T09:25:50.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2023-06-24T09:25:50.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:50.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-24T09:25:50.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "smithi045" 2023-06-24T09:25:50.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:50.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-06-24T09:25:50.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-24T09:25:50.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:50.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-06-24T09:25:50.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-24T09:25:50.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:50.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-24T09:25:50.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:50.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-24T09:25:50.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-24T09:25:50.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-24T09:25:50.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-24T09:25:50.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-24T09:25:50.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-24T09:25:50.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:50.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-24T09:25:50.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-24T09:25:50.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-24T09:25:50.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-24T09:25:50.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-24T09:25:50.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-24T09:25:50.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-24T09:25:50.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-24T09:25:50.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-24T09:25:50.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:50.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-24T09:25:50.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:50.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-24T09:25:50.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-24T09:25:50.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:50.641 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-24T09:25:50.641 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2023-06-24T09:25:50.641 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-24T09:25:50.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-24T09:25:50.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-24T09:25:50.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-24T09:25:50.642 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2023-06-24T09:25:50.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:50.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-24T09:25:50.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:50.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-24T09:25:50.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:50.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2023-06-24T09:25:36.937607+0000", 2023-06-24T09:25:50.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-24T09:25:50.644 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:50.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-24T09:25:50.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-06-24T09:25:50.645 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (3/15)... 2023-06-24T09:25:50.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:50 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2704542122' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-24T09:25:50.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:50 smithi045 ceph-mon[102509]: mgrmap e3: smithi045.dcgntd(active, since 1.00572s) 2023-06-24T09:25:53.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:53 smithi045 ceph-mon[102509]: mgrmap e4: smithi045.dcgntd(active, since 2s) 2023-06-24T09:25:53.747 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:53.747 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-06-24T09:25:53.747 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "fceaf654-1270-11ee-9b2b-001a4aab830c", 2023-06-24T09:25:53.747 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2023-06-24T09:25:53.748 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-24T09:25:53.748 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-24T09:25:53.748 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-24T09:25:53.748 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:53.748 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-24T09:25:53.748 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-24T09:25:53.748 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2023-06-24T09:25:53.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:53.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-24T09:25:53.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "smithi045" 2023-06-24T09:25:53.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:53.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-06-24T09:25:53.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-24T09:25:53.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:53.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-06-24T09:25:53.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-24T09:25:53.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:53.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-24T09:25:53.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:53.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-24T09:25:53.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-24T09:25:53.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-24T09:25:53.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-24T09:25:53.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-24T09:25:53.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-24T09:25:53.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:53.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-24T09:25:53.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-24T09:25:53.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-24T09:25:53.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-24T09:25:53.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-24T09:25:53.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-24T09:25:53.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-24T09:25:53.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-24T09:25:53.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-24T09:25:53.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:53.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-24T09:25:53.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:53.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-24T09:25:53.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-24T09:25:53.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:53.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-24T09:25:53.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-24T09:25:53.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-24T09:25:53.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-24T09:25:53.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-24T09:25:53.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-24T09:25:53.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2023-06-24T09:25:53.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2023-06-24T09:25:53.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-24T09:25:53.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:53.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-24T09:25:53.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-24T09:25:53.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2023-06-24T09:25:36.937607+0000", 2023-06-24T09:25:53.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-24T09:25:53.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2023-06-24T09:25:53.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-24T09:25:53.758 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-06-24T09:25:53.758 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2023-06-24T09:25:54.377 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:54 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/559190980' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-24T09:25:54.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:54.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2023-06-24T09:25:54.723 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:25:54.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-06-24T09:25:54.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-06-24T09:25:54.724 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-06-24T09:25:54.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-06-24T09:25:54.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:54.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2023-06-24T09:25:54.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-06-24T09:25:54.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2023-06-24T09:25:54.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2023-06-24T09:25:54.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-06-24T09:25:54.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-06-24T09:25:54.727 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2023-06-24T09:25:55.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:55 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2806812474' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-06-24T09:25:56.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:56 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3632373150' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-06-24T09:25:57.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:57 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3632373150' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-06-24T09:25:57.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:25:57 smithi045 ceph-mon[102509]: mgrmap e5: smithi045.dcgntd(active, since 6s) 2023-06-24T09:26:00.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:00 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1953604755' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-06-24T09:26:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-06-24T09:26:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-06-24T09:26:02.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-24T09:26:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "smithi045.dcgntd", 2023-06-24T09:26:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-06-24T09:26:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-06-24T09:26:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2023-06-24T09:26:02.545 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2023-06-24T09:26:03.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: Active manager daemon smithi045.dcgntd restarted 2023-06-24T09:26:03.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: Activating manager daemon smithi045.dcgntd 2023-06-24T09:26:03.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: osdmap e2: 0 total, 0 up, 0 in 2023-06-24T09:26:03.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: mgrmap e6: smithi045.dcgntd(active, starting, since 0.0041193s) 2023-06-24T09:26:03.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi045"}]: dispatch 2023-06-24T09:26:03.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr metadata", "who": "smithi045.dcgntd", "id": "smithi045.dcgntd"}]: dispatch 2023-06-24T09:26:03.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-24T09:26:03.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-24T09:26:03.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-24T09:26:03.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: Manager daemon smithi045.dcgntd is now available 2023-06-24T09:26:03.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:03.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:03.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:26:03.175 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:02 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:26:04.260 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-06-24T09:26:04.261 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-06-24T09:26:04.261 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2023-06-24T09:26:04.261 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-06-24T09:26:04.261 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2023-06-24T09:26:04.261 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2023-06-24T09:26:04.263 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:03 smithi045 ceph-mon[102509]: Found migration_current of "None". Setting to last migration. 2023-06-24T09:26:04.263 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:03 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:26:04.263 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:03 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi045.dcgntd/mirror_snapshot_schedule"}]: dispatch 2023-06-24T09:26:04.263 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:03 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi045.dcgntd/trash_purge_schedule"}]: dispatch 2023-06-24T09:26:04.264 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:03 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:04.264 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:03 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:04.264 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:03 smithi045 ceph-mon[102509]: mgrmap e7: smithi045.dcgntd(active, since 1.00508s) 2023-06-24T09:26:05.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:04 smithi045 ceph-mon[102509]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-06-24T09:26:05.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:04 smithi045 ceph-mon[102509]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-06-24T09:26:05.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:04 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:05.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:04 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:26:06.110 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2023-06-24T09:26:06.110 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2023-06-24T09:26:06.112 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:05 smithi045 ceph-mon[102509]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:06.113 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:05 smithi045 ceph-mon[102509]: [24/Jun/2023:09:26:05] ENGINE Bus STARTING 2023-06-24T09:26:06.113 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:05 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:26:06.113 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:05 smithi045 ceph-mon[102509]: mgrmap e8: smithi045.dcgntd(active, since 2s) 2023-06-24T09:26:07.312 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:07 smithi045 ceph-mon[102509]: [24/Jun/2023:09:26:05] ENGINE Serving on https://172.21.15.45:7150 2023-06-24T09:26:07.312 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:07 smithi045 ceph-mon[102509]: [24/Jun/2023:09:26:05] ENGINE Serving on http://172.21.15.45:8765 2023-06-24T09:26:07.313 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:07 smithi045 ceph-mon[102509]: [24/Jun/2023:09:26:05] ENGINE Bus STARTED 2023-06-24T09:26:07.313 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:07 smithi045 ceph-mon[102509]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:07.313 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:07 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:07.313 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:07 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:08.245 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:08 smithi045 ceph-mon[102509]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:08.246 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:08 smithi045 ceph-mon[102509]: Generating ssh key... 2023-06-24T09:26:08.251 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDsjCBiK/VKuFB91ziXxCGjJBzqR75JL4hMMUgJlCpNinfAo5+tiRqdVOgBV5zCG851BSRoZ7mPDmDDmym3TMFiy9KajvWN90GbYbsAQJrYnm9yyO2KO9GBC9X32rQSSaqD7Uhuqhhrj9pY5En5cmNMWhfDYmLOhxs8a+lsVo7L8DhDvPJC18pBTWsMYI3bNHcVR336tD+eN+7Hn5rJjguoFhz/dc56CrYo2j/dkgLhVT3m5gz3evXxr4ItM87Zv2oxs+7rZTNRLlhOuzCRkvWsNdPhUjADvoqpojqoYTa7qxE7jXRkq8ri0Yfu0vIw0sSoeAam2TYVA7hOdjh+Pgm7TCdk6oP2ZB5v2tz0oH7lJtdTBBeY24JsV2SrSOBzPQgFCLwuL8YzRjja0q7ARYWd4oMIPAXi2OC7MUof7MAfeo2d7XFjkqeND9V8o+fhxQWwUeP6YnkrwJg2/x32AHq/ASViN2PzYamBYvJN45XuI8wPeMO6Gpm4YXKWPd0YpY0= ceph-fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:26:08.252 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-06-24T09:26:08.252 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2023-06-24T09:26:08.253 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2023-06-24T09:26:09.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:09 smithi045 ceph-mon[102509]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:10.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:10 smithi045 ceph-mon[102509]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:11.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2023-06-24T09:26:11.359 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying mon service with default placement... 2023-06-24T09:26:11.362 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:11 smithi045 ceph-mon[102509]: Deploying cephadm binary to smithi045 2023-06-24T09:26:11.363 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:11 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:12.138 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:12 smithi045 ceph-mon[102509]: Added host smithi045 2023-06-24T09:26:12.138 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:12 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:26:12.139 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:12 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:12.592 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-06-24T09:26:12.593 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying mgr service with default placement... 2023-06-24T09:26:13.408 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:13 smithi045 ceph-mon[102509]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:13.408 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:13 smithi045 ceph-mon[102509]: Saving service mon spec with placement count:5 2023-06-24T09:26:13.408 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:13 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:13.585 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-06-24T09:26:13.585 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying crash service with default placement... 2023-06-24T09:26:14.500 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:14 smithi045 ceph-mon[102509]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:14.500 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:14 smithi045 ceph-mon[102509]: Saving service mgr spec with placement count:2 2023-06-24T09:26:14.500 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:14 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:14.501 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:14 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:14.501 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:14 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:14.659 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2023-06-24T09:26:14.659 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying ceph-exporter service with default placement... 2023-06-24T09:26:15.611 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:15 smithi045 ceph-mon[102509]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:15.611 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:15 smithi045 ceph-mon[102509]: Saving service crash spec with placement * 2023-06-24T09:26:15.611 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:15 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:15.612 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:15 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:15.612 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:15 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:16.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2023-06-24T09:26:16.184 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying prometheus service with default placement... 2023-06-24T09:26:17.135 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:17 smithi045 ceph-mon[102509]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:17.135 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:17 smithi045 ceph-mon[102509]: Saving service ceph-exporter spec with placement * 2023-06-24T09:26:17.135 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:17 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2023-06-24T09:26:17.192 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying grafana service with default placement... 2023-06-24T09:26:18.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:18 smithi045 ceph-mon[102509]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:18.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:18 smithi045 ceph-mon[102509]: Saving service prometheus spec with placement count:1 2023-06-24T09:26:18.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:18 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:18.375 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2023-06-24T09:26:18.375 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying node-exporter service with default placement... 2023-06-24T09:26:19.342 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2023-06-24T09:26:19.342 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying alertmanager service with default placement... 2023-06-24T09:26:19.344 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:19 smithi045 ceph-mon[102509]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:19.344 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:19 smithi045 ceph-mon[102509]: Saving service grafana spec with placement count:1 2023-06-24T09:26:19.344 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:19 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:20.399 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2023-06-24T09:26:20.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:20 smithi045 ceph-mon[102509]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:20.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:20 smithi045 ceph-mon[102509]: Saving service node-exporter spec with placement * 2023-06-24T09:26:20.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:20 smithi045 ceph-mon[102509]: from='mgr.14120 172.21.15.45:0/945726897' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:21.344 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:21 smithi045 ceph-mon[102509]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:21.344 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:21 smithi045 ceph-mon[102509]: Saving service alertmanager spec with placement count:1 2023-06-24T09:26:21.344 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:21 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2406699083' entity='client.admin' 2023-06-24T09:26:22.324 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2023-06-24T09:26:23.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:22 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1774568945' entity='client.admin' 2023-06-24T09:26:24.337 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:23 smithi045 ceph-mon[102509]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:26:24.337 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:23 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/836807145' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-06-24T09:26:25.343 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:24 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/836807145' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-06-24T09:26:25.343 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:24 smithi045 ceph-mon[102509]: mgrmap e9: smithi045.dcgntd(active, since 21s) 2023-06-24T09:26:25.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-06-24T09:26:25.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-06-24T09:26:25.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-24T09:26:25.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "smithi045.dcgntd", 2023-06-24T09:26:25.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-06-24T09:26:25.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-06-24T09:26:25.368 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2023-06-24T09:26:25.369 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2023-06-24T09:26:26.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:25 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/387808749' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-06-24T09:26:31.007 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: Active manager daemon smithi045.dcgntd restarted 2023-06-24T09:26:31.007 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: Activating manager daemon smithi045.dcgntd 2023-06-24T09:26:31.007 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: osdmap e3: 0 total, 0 up, 0 in 2023-06-24T09:26:31.007 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: mgrmap e10: smithi045.dcgntd(active, starting, since 0.00352192s) 2023-06-24T09:26:31.007 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi045"}]: dispatch 2023-06-24T09:26:31.008 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr metadata", "who": "smithi045.dcgntd", "id": "smithi045.dcgntd"}]: dispatch 2023-06-24T09:26:31.008 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-24T09:26:31.008 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-24T09:26:31.008 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-24T09:26:31.008 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: Manager daemon smithi045.dcgntd is now available 2023-06-24T09:26:31.008 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:26:31.009 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:30 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:26:32.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2023-06-24T09:26:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-06-24T09:26:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2023-06-24T09:26:32.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2023-06-24T09:26:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2023-06-24T09:26:32.267 INFO:teuthology.orchestra.run.smithi045.stdout:Generating a dashboard self-signed certificate... 2023-06-24T09:26:32.268 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:31 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi045.dcgntd/mirror_snapshot_schedule"}]: dispatch 2023-06-24T09:26:32.268 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:31 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi045.dcgntd/trash_purge_schedule"}]: dispatch 2023-06-24T09:26:32.268 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:31 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:32.268 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:31 smithi045 ceph-mon[102509]: mgrmap e11: smithi045.dcgntd(active, since 1.0057s) 2023-06-24T09:26:33.212 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:32 smithi045 ceph-mon[102509]: [24/Jun/2023:09:26:31] ENGINE Bus STARTING 2023-06-24T09:26:33.213 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:32 smithi045 ceph-mon[102509]: [24/Jun/2023:09:26:31] ENGINE Serving on http://172.21.15.45:8765 2023-06-24T09:26:33.213 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:32 smithi045 ceph-mon[102509]: [24/Jun/2023:09:26:31] ENGINE Serving on https://172.21.15.45:7150 2023-06-24T09:26:33.213 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:32 smithi045 ceph-mon[102509]: [24/Jun/2023:09:26:31] ENGINE Bus STARTED 2023-06-24T09:26:33.213 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:32 smithi045 ceph-mon[102509]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-06-24T09:26:33.213 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:32 smithi045 ceph-mon[102509]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-06-24T09:26:33.213 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:32 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:33.514 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-06-24T09:26:33.514 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2023-06-24T09:26:34.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:33 smithi045 ceph-mon[102509]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:34.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:33 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:34.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:33 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:34.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:33 smithi045 ceph-mon[102509]: mgrmap e12: smithi045.dcgntd(active, since 2s) 2023-06-24T09:26:36.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$BrYrIWiAWmZIkdLqDDKm3exFx.dMDrDzN6FpLM/kxy2Dj4BkBs3xK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1687598795, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-06-24T09:26:36.773 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2023-06-24T09:26:36.776 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:36 smithi045 ceph-mon[102509]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:36.776 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:36 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:37.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:37 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2105357970' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-06-24T09:26:37.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2023-06-24T09:26:37.714 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-24T09:26:37.715 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2023-06-24T09:26:37.722 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-24T09:26:37.722 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2023-06-24T09:26:37.723 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2023-06-24T09:26:37.723 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-06-24T09:26:37.724 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2023-06-24T09:26:37.724 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:26:37.724 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045.front.sepia.ceph.com:8443/ 2023-06-24T09:26:37.724 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2023-06-24T09:26:37.725 INFO:teuthology.orchestra.run.smithi045.stdout: Password: 2tbzid9huc 2023-06-24T09:26:37.725 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:26:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config directory 2023-06-24T09:26:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling autotune for osd_memory_target 2023-06-24T09:26:38.657 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:38 smithi045 ceph-mon[102509]: mgrmap e13: smithi045.dcgntd(active, since 6s) 2023-06-24T09:26:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-06-24T09:26:39.738 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-06-24T09:26:39.739 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:26:39.739 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-06-24T09:26:39.739 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:26:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2023-06-24T09:26:39.739 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:26:39.739 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-06-24T09:26:39.739 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:26:39.739 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2023-06-24T09:26:39.740 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:26:39.740 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2023-06-24T09:26:39.740 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:26:39.740 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2023-06-24T09:26:39.740 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:26:39.740 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-06-24T09:26:39.740 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:26:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2023-06-24T09:26:39.777 INFO:tasks.cephadm:Fetching config... 2023-06-24T09:26:39.777 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:26:39.777 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-06-24T09:26:39.797 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-06-24T09:26:39.797 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:26:39.797 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-06-24T09:26:39.856 INFO:tasks.cephadm:Fetching mon keyring... 2023-06-24T09:26:39.856 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:26:39.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/keyring of=/dev/stdout 2023-06-24T09:26:39.925 INFO:tasks.cephadm:Fetching pub ssh key... 2023-06-24T09:26:39.925 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:26:39.925 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-06-24T09:26:39.983 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-06-24T09:26:39.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDsjCBiK/VKuFB91ziXxCGjJBzqR75JL4hMMUgJlCpNinfAo5+tiRqdVOgBV5zCG851BSRoZ7mPDmDDmym3TMFiy9KajvWN90GbYbsAQJrYnm9yyO2KO9GBC9X32rQSSaqD7Uhuqhhrj9pY5En5cmNMWhfDYmLOhxs8a+lsVo7L8DhDvPJC18pBTWsMYI3bNHcVR336tD+eN+7Hn5rJjguoFhz/dc56CrYo2j/dkgLhVT3m5gz3evXxr4ItM87Zv2oxs+7rZTNRLlhOuzCRkvWsNdPhUjADvoqpojqoYTa7qxE7jXRkq8ri0Yfu0vIw0sSoeAam2TYVA7hOdjh+Pgm7TCdk6oP2ZB5v2tz0oH7lJtdTBBeY24JsV2SrSOBzPQgFCLwuL8YzRjja0q7ARYWd4oMIPAXi2OC7MUof7MAfeo2d7XFjkqeND9V8o+fhxQWwUeP6YnkrwJg2/x32AHq/ASViN2PzYamBYvJN45XuI8wPeMO6Gpm4YXKWPd0YpY0= ceph-fceaf654-1270-11ee-9b2b-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-06-24T09:26:40.062 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDsjCBiK/VKuFB91ziXxCGjJBzqR75JL4hMMUgJlCpNinfAo5+tiRqdVOgBV5zCG851BSRoZ7mPDmDDmym3TMFiy9KajvWN90GbYbsAQJrYnm9yyO2KO9GBC9X32rQSSaqD7Uhuqhhrj9pY5En5cmNMWhfDYmLOhxs8a+lsVo7L8DhDvPJC18pBTWsMYI3bNHcVR336tD+eN+7Hn5rJjguoFhz/dc56CrYo2j/dkgLhVT3m5gz3evXxr4ItM87Zv2oxs+7rZTNRLlhOuzCRkvWsNdPhUjADvoqpojqoYTa7qxE7jXRkq8ri0Yfu0vIw0sSoeAam2TYVA7hOdjh+Pgm7TCdk6oP2ZB5v2tz0oH7lJtdTBBeY24JsV2SrSOBzPQgFCLwuL8YzRjja0q7ARYWd4oMIPAXi2OC7MUof7MAfeo2d7XFjkqeND9V8o+fhxQWwUeP6YnkrwJg2/x32AHq/ASViN2PzYamBYvJN45XuI8wPeMO6Gpm4YXKWPd0YpY0= ceph-fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:26:40.073 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDsjCBiK/VKuFB91ziXxCGjJBzqR75JL4hMMUgJlCpNinfAo5+tiRqdVOgBV5zCG851BSRoZ7mPDmDDmym3TMFiy9KajvWN90GbYbsAQJrYnm9yyO2KO9GBC9X32rQSSaqD7Uhuqhhrj9pY5En5cmNMWhfDYmLOhxs8a+lsVo7L8DhDvPJC18pBTWsMYI3bNHcVR336tD+eN+7Hn5rJjguoFhz/dc56CrYo2j/dkgLhVT3m5gz3evXxr4ItM87Zv2oxs+7rZTNRLlhOuzCRkvWsNdPhUjADvoqpojqoYTa7qxE7jXRkq8ri0Yfu0vIw0sSoeAam2TYVA7hOdjh+Pgm7TCdk6oP2ZB5v2tz0oH7lJtdTBBeY24JsV2SrSOBzPQgFCLwuL8YzRjja0q7ARYWd4oMIPAXi2OC7MUof7MAfeo2d7XFjkqeND9V8o+fhxQWwUeP6YnkrwJg2/x32AHq/ASViN2PzYamBYvJN45XuI8wPeMO6Gpm4YXKWPd0YpY0= ceph-fceaf654-1270-11ee-9b2b-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-06-24T09:26:40.119 INFO:teuthology.orchestra.run.smithi188.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDsjCBiK/VKuFB91ziXxCGjJBzqR75JL4hMMUgJlCpNinfAo5+tiRqdVOgBV5zCG851BSRoZ7mPDmDDmym3TMFiy9KajvWN90GbYbsAQJrYnm9yyO2KO9GBC9X32rQSSaqD7Uhuqhhrj9pY5En5cmNMWhfDYmLOhxs8a+lsVo7L8DhDvPJC18pBTWsMYI3bNHcVR336tD+eN+7Hn5rJjguoFhz/dc56CrYo2j/dkgLhVT3m5gz3evXxr4ItM87Zv2oxs+7rZTNRLlhOuzCRkvWsNdPhUjADvoqpojqoYTa7qxE7jXRkq8ri0Yfu0vIw0sSoeAam2TYVA7hOdjh+Pgm7TCdk6oP2ZB5v2tz0oH7lJtdTBBeY24JsV2SrSOBzPQgFCLwuL8YzRjja0q7ARYWd4oMIPAXi2OC7MUof7MAfeo2d7XFjkqeND9V8o+fhxQWwUeP6YnkrwJg2/x32AHq/ASViN2PzYamBYvJN45XuI8wPeMO6Gpm4YXKWPd0YpY0= ceph-fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:26:40.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-06-24T09:26:40.373 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:26:40.651 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:40 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1760468893' entity='client.admin' 2023-06-24T09:26:42.463 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-06-24T09:26:42.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-06-24T09:26:42.706 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:26:43.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:43 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/133468251' entity='client.admin' 2023-06-24T09:26:44.137 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:44 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:44.137 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:44 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:44.137 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:44 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:26:44.137 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:44 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:44.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:44 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi045", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2023-06-24T09:26:44.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:44 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi045", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2023-06-24T09:26:44.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:44 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:26:44.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:44 smithi045 ceph-mon[102509]: Deploying daemon ceph-exporter.smithi045 on smithi045 2023-06-24T09:26:45.201 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi188 2023-06-24T09:26:45.201 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-06-24T09:26:45.202 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.conf 2023-06-24T09:26:45.223 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-06-24T09:26:45.223 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-24T09:26:45.284 INFO:tasks.cephadm:Adding host smithi188 to orchestrator... 2023-06-24T09:26:45.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch host add smithi188 2023-06-24T09:26:45.630 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:26:46.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:45 smithi045 ceph-mon[102509]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:46.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:46.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:46.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:46.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:46.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:46.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi045", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2023-06-24T09:26:46.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi045", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2023-06-24T09:26:46.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:26:47.267 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:46 smithi045 ceph-mon[102509]: Deploying daemon crash.smithi045 on smithi045 2023-06-24T09:26:47.267 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:46 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:49.319 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:48 smithi045 ceph-mon[102509]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:49.320 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:48 smithi045 ceph-mon[102509]: Deploying cephadm binary to smithi188 2023-06-24T09:26:49.320 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:48 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:49.320 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:48 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:49.321 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:48 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:49.321 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:48 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:49.321 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:48 smithi045 ceph-mon[102509]: Deploying daemon node-exporter.smithi045 on smithi045 2023-06-24T09:26:49.411 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi188' with addr '172.21.15.188' 2023-06-24T09:26:50.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch host ls --format=json 2023-06-24T09:26:50.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:50 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:50.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:50 smithi045 ceph-mon[102509]: Added host smithi188 2023-06-24T09:26:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:26:52.139 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:51 smithi045 ceph-mon[102509]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:26:52.139 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:51 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:52.540 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:26:52.540 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.188", "hostname": "smithi188", "labels": [], "status": ""}] 2023-06-24T09:26:52.908 INFO:tasks.cephadm:Setting crush tunables to default 2023-06-24T09:26:52.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd crush tunables default 2023-06-24T09:26:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:26:53.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:52 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:53.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:52 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:53.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:52 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:53.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:52 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:53.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:52 smithi045 ceph-mon[102509]: Deploying daemon alertmanager.smithi045 on smithi045 2023-06-24T09:26:54.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:53 smithi045 ceph-mon[102509]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:26:54.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:53 smithi045 ceph-mon[102509]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:26:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2023-06-24T09:26:56.284 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:55 smithi045 ceph-mon[102509]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:26:56.284 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:55 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2584128042' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-06-24T09:26:56.284 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:55 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:56.750 INFO:tasks.cephadm:Adding mon.smithi045 on smithi045 2023-06-24T09:26:56.750 INFO:tasks.cephadm:Adding mon.smithi188 on smithi188 2023-06-24T09:26:56.751 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch apply mon '2;smithi045:172.21.15.45=smithi045;smithi188:172.21.15.188=smithi188' 2023-06-24T09:26:57.011 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:26:57.078 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:26:57.405 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:56 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2584128042' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-06-24T09:26:57.405 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:56 smithi045 ceph-mon[102509]: osdmap e4: 0 total, 0 up, 0 in 2023-06-24T09:26:58.104 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:57 smithi045 ceph-mon[102509]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:26:58.576 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mon update... 2023-06-24T09:26:58.974 DEBUG:teuthology.orchestra.run.smithi188:mon.smithi188> sudo journalctl -f -n 0 -u ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mon.smithi188.service 2023-06-24T09:26:58.976 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:26:58.976 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:26:59.006 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:-- Logs begin at Sat 2023-06-24 09:16:51 UTC. -- 2023-06-24T09:26:59.247 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:26:59.310 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:26:59.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi045:172.21.15.45=smithi045;smithi188:172.21.15.188=smithi188", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:26:59.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: Saving service mon spec with placement smithi045:172.21.15.45=smithi045;smithi188:172.21.15.188=smithi188;count:2 2023-06-24T09:26:59.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:59.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:59.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:59.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:59.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:59.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: Regenerating cephadm self-signed grafana TLS certificates 2023-06-24T09:26:59.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:59.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:59.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-06-24T09:26:59.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-06-24T09:26:59.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:26:59.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: Deploying daemon grafana.smithi045 on smithi045 2023-06-24T09:26:59.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:26:59 smithi045 ceph-mon[102509]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:00.693 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:00.693 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:00.696 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:00.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:00 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/1074216251' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:02.107 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:02.108 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:02.366 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:02.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:01 smithi045 ceph-mon[102509]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:02.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:01 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:02.432 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:03.788 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:03.788 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:03.790 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:04.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:03 smithi045 ceph-mon[102509]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:04.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:03 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/4179491130' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:05.178 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:05.179 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:05.434 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:05.501 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:06.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:05 smithi045 ceph-mon[102509]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:06.807 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:06.808 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:06.809 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:07.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:07 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/1489159888' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:07.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:07 smithi045 ceph-mon[102509]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:08.251 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:08.252 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:08.503 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:08.568 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:10.000 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:10.000 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:10.002 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:10.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:09 smithi045 ceph-mon[102509]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:11.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:10 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/3377918133' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:11.429 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:11.430 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:11.690 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:11.757 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:12.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:11 smithi045 ceph-mon[102509]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:13.151 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:13.151 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:13.153 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:14.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:13 smithi045 ceph-mon[102509]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:14.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:13 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/2782195661' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:14.617 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:14.617 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:14.876 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:14.940 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:16.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:15 smithi045 ceph-mon[102509]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:16.471 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:16.472 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:16.473 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:17.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:17 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/4246973589' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:17.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:17 smithi045 ceph-mon[102509]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:17.877 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:17.878 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:18.131 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:18.196 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:19.606 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:19.606 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:19.608 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:20.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:19 smithi045 ceph-mon[102509]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:20.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:19 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/2681738066' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:21.035 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:21.035 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:21.283 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:21.355 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:22.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:21 smithi045 ceph-mon[102509]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:22.759 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:22.760 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:22.761 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:23.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:22 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/2529990354' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:24.245 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:24.246 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:24.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:23 smithi045 ceph-mon[102509]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:24.505 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:24.576 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:25.996 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:25.997 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:25.998 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:26.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:25 smithi045 ceph-mon[102509]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:27.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:26 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/3310106427' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:27.462 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:27.463 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:27.720 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:27.789 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:28.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:27 smithi045 ceph-mon[102509]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:29.220 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:29.220 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:29.222 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:30.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:29 smithi045 ceph-mon[102509]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:30.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:29 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/805695541' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:30.653 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:30.654 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:30.910 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:30.977 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:32.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:31 smithi045 ceph-mon[102509]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:32.423 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:32.423 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:32.425 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:33.013 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:32 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/3717983472' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:33.856 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:33.857 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:34.120 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:34.126 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:33 smithi045 ceph-mon[102509]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:34.188 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:35.306 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:35 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:35.307 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:35 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:35.307 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:35 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:35.307 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:35 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:35.307 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:35 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:35.308 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:35 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:35.308 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:35 smithi045 ceph-mon[102509]: Deploying daemon prometheus.smithi045 on smithi045 2023-06-24T09:27:35.308 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:35 smithi045 ceph-mon[102509]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:35.573 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:35.575 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:36.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:36 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/748775731' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:36.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:36 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:37.051 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:37.051 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:37.303 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:37.369 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:37.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:37 smithi045 ceph-mon[102509]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:38.751 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:38.751 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:38.753 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:39.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:38 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/3558989343' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:40.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:39 smithi045 ceph-mon[102509]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:40.216 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:40.217 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:40.477 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:40.547 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:41.931 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:41.931 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:41.933 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:42.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:41 smithi045 ceph-mon[102509]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:42.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:41 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/421924713' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:43.329 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:43.330 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:43.582 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:43.644 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:44.364 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:43 smithi045 ceph-mon[102509]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:45.071 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:45.071 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:45.073 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:46.249 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:45 smithi045 ceph-mon[102509]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:27:46.249 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:46.249 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:46.250 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:46.250 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:45 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2023-06-24T09:27:46.250 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:45 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/2507922688' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:46.513 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:46.514 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:46.762 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:46.831 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:47.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:46 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:47.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:46 smithi045 ceph-mon[102509]: from='mgr.14164 172.21.15.45:0/621290705' entity='mgr.smithi045.dcgntd' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2023-06-24T09:27:47.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:46 smithi045 ceph-mon[102509]: mgrmap e14: smithi045.dcgntd(active, since 75s) 2023-06-24T09:27:48.254 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:48.254 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:48.256 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:48.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:48 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/834943321' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:49.686 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:49.686 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:49.950 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:50.017 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:51.448 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:51.448 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:51.450 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:52.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:52 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/2245728621' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:52.870 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:52.870 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:53.113 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: Active manager daemon smithi045.dcgntd restarted 2023-06-24T09:27:53.113 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: Activating manager daemon smithi045.dcgntd 2023-06-24T09:27:53.113 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: osdmap e5: 0 total, 0 up, 0 in 2023-06-24T09:27:53.113 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: mgrmap e15: smithi045.dcgntd(active, starting, since 0.00322637s) 2023-06-24T09:27:53.114 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi045"}]: dispatch 2023-06-24T09:27:53.114 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr metadata", "who": "smithi045.dcgntd", "id": "smithi045.dcgntd"}]: dispatch 2023-06-24T09:27:53.114 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-24T09:27:53.114 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-24T09:27:53.115 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-24T09:27:53.115 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: Manager daemon smithi045.dcgntd is now available 2023-06-24T09:27:53.115 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:27:53.116 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:27:53.116 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:53.116 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:27:53.116 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi045.dcgntd/mirror_snapshot_schedule"}]: dispatch 2023-06-24T09:27:53.117 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi045.dcgntd/trash_purge_schedule"}]: dispatch 2023-06-24T09:27:53.136 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:53.207 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:54.664 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:54.665 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:54.668 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:54.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:54 smithi045 ceph-mon[102509]: [24/Jun/2023:09:27:53] ENGINE Bus STARTING 2023-06-24T09:27:54.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:54 smithi045 ceph-mon[102509]: [24/Jun/2023:09:27:53] ENGINE Serving on https://172.21.15.45:7150 2023-06-24T09:27:54.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:54 smithi045 ceph-mon[102509]: [24/Jun/2023:09:27:53] ENGINE Serving on http://172.21.15.45:8765 2023-06-24T09:27:54.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:54 smithi045 ceph-mon[102509]: [24/Jun/2023:09:27:53] ENGINE Bus STARTED 2023-06-24T09:27:54.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:54 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:54.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:54 smithi045 ceph-mon[102509]: mgrmap e16: smithi045.dcgntd(active, since 1.00498s) 2023-06-24T09:27:54.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:54 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:55.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:55 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/757907257' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:55.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:55 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:56.097 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:56.098 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:27:56.328 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:56.383 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2023-06-24T09:27:57.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:56 smithi045 ceph-mon[102509]: mgrmap e17: smithi045.dcgntd(active, since 2s) 2023-06-24T09:27:57.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:56 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:57.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:56 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:58.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:57 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:58.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:57 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:58.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:57 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:27:58.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:57 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:58.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:57 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:58.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:57 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:58.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:57 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:58.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:57 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:27:58.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:57 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:27:58.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:57 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:27:58.256 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:27:58.256 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:27:58.259 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:27:59.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: Updating smithi045:/etc/ceph/ceph.conf 2023-06-24T09:27:59.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: Updating smithi188:/etc/ceph/ceph.conf 2023-06-24T09:27:59.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: Updating smithi188:/var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-24T09:27:59.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: Updating smithi045:/var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-24T09:27:59.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/2075543954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:27:59.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:59.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:59.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:59.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:59.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:27:59.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi188", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2023-06-24T09:27:59.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi188", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2023-06-24T09:27:59.175 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:27:59.718 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:27:59.718 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:28:00.078 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-24T09:28:00.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:59 smithi045 ceph-mon[102509]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2023-06-24T09:28:00.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:59 smithi045 ceph-mon[102509]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2023-06-24T09:28:00.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:59 smithi045 ceph-mon[102509]: Updating smithi188:/var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-24T09:28:00.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:59 smithi045 ceph-mon[102509]: Updating smithi045:/var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-24T09:28:00.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:27:59 smithi045 ceph-mon[102509]: Deploying daemon ceph-exporter.smithi188 on smithi188 2023-06-24T09:28:01.591 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:28:01.592 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:28:01.594 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:28:02.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:02.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:02.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:02.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:02.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi188", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2023-06-24T09:28:02.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi188", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2023-06-24T09:28:02.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:02.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:02 smithi045 ceph-mon[102509]: Deploying daemon crash.smithi188 on smithi188 2023-06-24T09:28:02.673 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:02 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/3189747288' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:28:03.029 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:28:03.030 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:28:03.473 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-24T09:28:04.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:04 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:04.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:04 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:04.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:04 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:04.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:04 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:04.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:04 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:04.988 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:28:04.989 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:28:04.990 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:28:05.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:05 smithi045 ceph-mon[102509]: Deploying daemon node-exporter.smithi188 on smithi188 2023-06-24T09:28:05.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:05 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/2389529030' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:28:06.497 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:28:06.498 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:28:06.924 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-24T09:28:08.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:07 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:08.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:07 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:08.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:07 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:08.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:07 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:08.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:07 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi188.zipfzj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-24T09:28:08.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:07 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi188.zipfzj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-24T09:28:08.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:07 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-24T09:28:08.424 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:07 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:08.424 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:07 smithi045 ceph-mon[102509]: Deploying daemon mgr.smithi188.zipfzj on smithi188 2023-06-24T09:28:08.933 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:28:08.934 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:25:34.820744Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-24T09:28:08.935 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2023-06-24T09:28:09.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:28:09.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:09.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:08 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/798883289' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:28:10.490 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-06-24T09:28:10.491 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-24T09:28:10.722 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-24T09:28:10.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:10 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:10.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:10 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:10.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:10 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:10.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:10 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:10.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:10 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-24T09:28:10.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:10 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:10.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:10 smithi045 ceph-mon[102509]: Deploying daemon mon.smithi188 on smithi188 2023-06-24T09:28:12.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 podman[112282]: 2023-06-24T09:28:13.022 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: set uid:gid to 167:167 (ceph:ceph) 2023-06-24T09:28:13.023 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: ceph version 18.1.0-166-gc9d917e3 (c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8) reef (rc), process ceph-mon, pid 2 2023-06-24T09:28:13.023 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: pidfile_write: ignore empty --pid-file 2023-06-24T09:28:13.023 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: load: jerasure load: lrc 2023-06-24T09:28:13.023 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 bash[112282]: 9ba7f01a869df87b3991b1459ad856757caf24fd41d331fe9cb9e9678abbda34 2023-06-24T09:28:13.023 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: RocksDB version: 7.9.2 2023-06-24T09:28:13.023 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Git sha 0 2023-06-24T09:28:13.023 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Compile date 2023-06-23 16:06:32 2023-06-24T09:28:13.024 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: DB SUMMARY 2023-06-24T09:28:13.024 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: DB Session ID: DQ4HZW349XSZBIBH3JMS 2023-06-24T09:28:13.024 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: CURRENT file: CURRENT 2023-06-24T09:28:13.024 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: IDENTITY file: IDENTITY 2023-06-24T09:28:13.024 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-06-24T09:28:13.024 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi188/store.db dir, Total Num: 0, files: 2023-06-24T09:28:13.024 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi188/store.db: 000004.log size: 511 ; 2023-06-24T09:28:13.025 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.error_if_exists: 0 2023-06-24T09:28:13.025 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.create_if_missing: 0 2023-06-24T09:28:13.025 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.paranoid_checks: 1 2023-06-24T09:28:13.025 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.flush_verify_memtable_count: 1 2023-06-24T09:28:13.025 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-06-24T09:28:13.025 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-06-24T09:28:13.025 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.env: 0x5650ba645720 2023-06-24T09:28:13.026 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.fs: PosixFileSystem 2023-06-24T09:28:13.026 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.info_log: 0x5650bc3892a0 2023-06-24T09:28:13.026 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_file_opening_threads: 16 2023-06-24T09:28:13.026 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.statistics: (nil) 2023-06-24T09:28:13.026 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.use_fsync: 0 2023-06-24T09:28:13.026 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_log_file_size: 0 2023-06-24T09:28:13.026 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-06-24T09:28:13.026 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.log_file_time_to_roll: 0 2023-06-24T09:28:13.027 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.keep_log_file_num: 1000 2023-06-24T09:28:13.027 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.recycle_log_file_num: 0 2023-06-24T09:28:13.027 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.allow_fallocate: 1 2023-06-24T09:28:13.027 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.allow_mmap_reads: 0 2023-06-24T09:28:13.027 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.allow_mmap_writes: 0 2023-06-24T09:28:13.027 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.use_direct_reads: 0 2023-06-24T09:28:13.027 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-06-24T09:28:13.028 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.create_missing_column_families: 0 2023-06-24T09:28:13.028 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.db_log_dir: 2023-06-24T09:28:13.028 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.wal_dir: 2023-06-24T09:28:13.028 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.table_cache_numshardbits: 6 2023-06-24T09:28:13.028 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.WAL_ttl_seconds: 0 2023-06-24T09:28:13.028 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.WAL_size_limit_MB: 0 2023-06-24T09:28:13.029 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-06-24T09:28:13.029 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-06-24T09:28:13.029 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.is_fd_close_on_exec: 1 2023-06-24T09:28:13.029 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.advise_random_on_open: 1 2023-06-24T09:28:13.029 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.db_write_buffer_size: 0 2023-06-24T09:28:13.029 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.write_buffer_manager: 0x5650bb61a5a0 2023-06-24T09:28:13.029 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-06-24T09:28:13.029 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-06-24T09:28:13.030 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.use_adaptive_mutex: 0 2023-06-24T09:28:13.030 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.rate_limiter: (nil) 2023-06-24T09:28:13.030 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-06-24T09:28:13.030 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.wal_recovery_mode: 2 2023-06-24T09:28:13.030 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.enable_thread_tracking: 0 2023-06-24T09:28:13.030 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.enable_pipelined_write: 0 2023-06-24T09:28:13.030 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.unordered_write: 0 2023-06-24T09:28:13.031 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-06-24T09:28:13.031 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-06-24T09:28:13.031 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-06-24T09:28:13.031 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-06-24T09:28:13.031 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.row_cache: None 2023-06-24T09:28:13.031 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.wal_filter: None 2023-06-24T09:28:13.031 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-06-24T09:28:13.031 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.allow_ingest_behind: 0 2023-06-24T09:28:13.032 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.two_write_queues: 0 2023-06-24T09:28:13.032 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.manual_wal_flush: 0 2023-06-24T09:28:13.032 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.wal_compression: 0 2023-06-24T09:28:13.032 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.atomic_flush: 0 2023-06-24T09:28:13.032 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-06-24T09:28:13.032 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.persist_stats_to_disk: 0 2023-06-24T09:28:13.032 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.write_dbid_to_manifest: 0 2023-06-24T09:28:13.033 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.log_readahead_size: 0 2023-06-24T09:28:13.033 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-06-24T09:28:13.033 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.best_efforts_recovery: 0 2023-06-24T09:28:13.033 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-06-24T09:28:13.033 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-06-24T09:28:13.033 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.allow_data_in_errors: 0 2023-06-24T09:28:13.033 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.db_host_id: __hostname__ 2023-06-24T09:28:13.034 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.enforce_single_del_contracts: true 2023-06-24T09:28:13.034 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_background_jobs: 2 2023-06-24T09:28:13.034 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_background_compactions: -1 2023-06-24T09:28:13.034 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_subcompactions: 1 2023-06-24T09:28:13.034 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-06-24T09:28:13.034 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-06-24T09:28:13.034 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.delayed_write_rate : 16777216 2023-06-24T09:28:13.034 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_total_wal_size: 0 2023-06-24T09:28:13.035 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-06-24T09:28:13.035 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.stats_dump_period_sec: 600 2023-06-24T09:28:13.035 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.stats_persist_period_sec: 600 2023-06-24T09:28:13.035 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-06-24T09:28:13.035 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_open_files: -1 2023-06-24T09:28:13.035 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bytes_per_sync: 0 2023-06-24T09:28:13.035 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.wal_bytes_per_sync: 0 2023-06-24T09:28:13.036 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.strict_bytes_per_sync: 0 2023-06-24T09:28:13.036 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_readahead_size: 0 2023-06-24T09:28:13.036 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_background_flushes: -1 2023-06-24T09:28:13.036 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Compression algorithms supported: 2023-06-24T09:28:13.036 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-06-24T09:28:13.036 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: kZSTD supported: 0 2023-06-24T09:28:13.036 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: kXpressCompression supported: 0 2023-06-24T09:28:13.037 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: kLZ4HCCompression supported: 1 2023-06-24T09:28:13.037 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: kZlibCompression supported: 1 2023-06-24T09:28:13.037 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: kSnappyCompression supported: 1 2023-06-24T09:28:13.037 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: kLZ4Compression supported: 1 2023-06-24T09:28:13.037 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: kBZip2Compression supported: 0 2023-06-24T09:28:13.038 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Fast CRC32 supported: Supported on x86 2023-06-24T09:28:13.038 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: DMutex implementation: pthread_mutex_t 2023-06-24T09:28:13.038 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi188/store.db/MANIFEST-000005 2023-06-24T09:28:13.038 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-06-24T09:28:13.039 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-06-24T09:28:13.039 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.merge_operator: 2023-06-24T09:28:13.039 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_filter: None 2023-06-24T09:28:13.040 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_filter_factory: None 2023-06-24T09:28:13.040 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.sst_partitioner_factory: None 2023-06-24T09:28:13.040 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.memtable_factory: SkipListFactory 2023-06-24T09:28:13.040 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.table_factory: BlockBasedTable 2023-06-24T09:28:13.041 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5650bc3893a0) 2023-06-24T09:28:13.041 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: cache_index_and_filter_blocks: 1 2023-06-24T09:28:13.041 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-06-24T09:28:13.041 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-06-24T09:28:13.042 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: pin_top_level_index_and_filter: 1 2023-06-24T09:28:13.042 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: index_type: 0 2023-06-24T09:28:13.042 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: data_block_index_type: 0 2023-06-24T09:28:13.043 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: index_shortening: 1 2023-06-24T09:28:13.043 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: data_block_hash_table_util_ratio: 0.750000 2023-06-24T09:28:13.043 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: checksum: 4 2023-06-24T09:28:13.044 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: no_block_cache: 0 2023-06-24T09:28:13.044 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: block_cache: 0x5650bb69b1f0 2023-06-24T09:28:13.044 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: block_cache_name: BinnedLRUCache 2023-06-24T09:28:13.044 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: block_cache_options: 2023-06-24T09:28:13.044 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: capacity : 536870912 2023-06-24T09:28:13.044 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: num_shard_bits : 4 2023-06-24T09:28:13.044 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: strict_capacity_limit : 0 2023-06-24T09:28:13.045 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: high_pri_pool_ratio: 0.000 2023-06-24T09:28:13.045 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: block_cache_compressed: (nil) 2023-06-24T09:28:13.045 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: persistent_cache: (nil) 2023-06-24T09:28:13.045 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: block_size: 4096 2023-06-24T09:28:13.045 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: block_size_deviation: 10 2023-06-24T09:28:13.045 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: block_restart_interval: 16 2023-06-24T09:28:13.045 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: index_block_restart_interval: 1 2023-06-24T09:28:13.046 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: metadata_block_size: 4096 2023-06-24T09:28:13.046 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: partition_filters: 0 2023-06-24T09:28:13.046 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: use_delta_encoding: 1 2023-06-24T09:28:13.046 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: filter_policy: bloomfilter 2023-06-24T09:28:13.046 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: whole_key_filtering: 1 2023-06-24T09:28:13.046 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: verify_compression: 0 2023-06-24T09:28:13.046 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: read_amp_bytes_per_bit: 0 2023-06-24T09:28:13.046 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: format_version: 5 2023-06-24T09:28:13.047 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: enable_index_compression: 1 2023-06-24T09:28:13.047 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: block_align: 0 2023-06-24T09:28:13.047 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: max_auto_readahead_size: 262144 2023-06-24T09:28:13.047 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: prepopulate_block_cache: 0 2023-06-24T09:28:13.047 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: initial_auto_readahead_size: 8192 2023-06-24T09:28:13.047 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: num_file_reads_for_auto_readahead: 2 2023-06-24T09:28:13.047 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.write_buffer_size: 33554432 2023-06-24T09:28:13.048 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_write_buffer_number: 2 2023-06-24T09:28:13.048 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compression: NoCompression 2023-06-24T09:28:13.048 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bottommost_compression: Disabled 2023-06-24T09:28:13.048 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.prefix_extractor: nullptr 2023-06-24T09:28:13.048 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-06-24T09:28:13.049 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.num_levels: 7 2023-06-24T09:28:13.049 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-06-24T09:28:13.049 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-06-24T09:28:13.049 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-06-24T09:28:13.049 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-06-24T09:28:13.049 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-06-24T09:28:13.050 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-06-24T09:28:13.051 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-06-24T09:28:13.051 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-06-24T09:28:13.051 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-06-24T09:28:13.051 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-06-24T09:28:13.051 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-06-24T09:28:13.051 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-06-24T09:28:13.052 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compression_opts.window_bits: -14 2023-06-24T09:28:13.052 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compression_opts.level: 32767 2023-06-24T09:28:13.052 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compression_opts.strategy: 0 2023-06-24T09:28:13.052 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-06-24T09:28:13.052 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-06-24T09:28:13.052 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-06-24T09:28:13.052 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-06-24T09:28:13.053 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compression_opts.enabled: false 2023-06-24T09:28:13.053 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-06-24T09:28:13.053 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-06-24T09:28:13.053 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-06-24T09:28:13.053 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-06-24T09:28:13.053 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.target_file_size_base: 67108864 2023-06-24T09:28:13.053 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.target_file_size_multiplier: 1 2023-06-24T09:28:13.054 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-06-24T09:28:13.054 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-06-24T09:28:13.054 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-06-24T09:28:13.054 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-06-24T09:28:13.054 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-06-24T09:28:13.054 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-06-24T09:28:13.054 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-06-24T09:28:13.054 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-06-24T09:28:13.055 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-06-24T09:28:13.055 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-06-24T09:28:13.055 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-06-24T09:28:13.055 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-06-24T09:28:13.055 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-06-24T09:28:13.055 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.arena_block_size: 1048576 2023-06-24T09:28:13.055 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-06-24T09:28:13.055 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-06-24T09:28:13.056 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.disable_auto_compactions: 0 2023-06-24T09:28:13.056 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-06-24T09:28:13.056 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-06-24T09:28:13.056 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-06-24T09:28:13.056 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-06-24T09:28:13.056 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-06-24T09:28:13.056 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-06-24T09:28:13.057 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-06-24T09:28:13.057 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-06-24T09:28:13.057 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-06-24T09:28:13.057 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-06-24T09:28:13.057 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.table_properties_collectors: 2023-06-24T09:28:13.057 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.inplace_update_support: 0 2023-06-24T09:28:13.058 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.inplace_update_num_locks: 10000 2023-06-24T09:28:13.058 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-06-24T09:28:13.058 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-06-24T09:28:13.058 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.memtable_huge_page_size: 0 2023-06-24T09:28:13.058 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.bloom_locality: 0 2023-06-24T09:28:13.059 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.max_successive_merges: 0 2023-06-24T09:28:13.059 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.optimize_filters_for_hits: 0 2023-06-24T09:28:13.059 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.paranoid_file_checks: 0 2023-06-24T09:28:13.059 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.force_consistency_checks: 1 2023-06-24T09:28:13.059 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.report_bg_io_stats: 0 2023-06-24T09:28:13.060 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.ttl: 2592000 2023-06-24T09:28:13.060 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.periodic_compaction_seconds: 0 2023-06-24T09:28:13.060 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-06-24T09:28:13.060 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-06-24T09:28:13.061 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.enable_blob_files: false 2023-06-24T09:28:13.061 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.min_blob_size: 0 2023-06-24T09:28:13.061 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.blob_file_size: 268435456 2023-06-24T09:28:13.061 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.blob_compression_type: NoCompression 2023-06-24T09:28:13.061 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.enable_blob_garbage_collection: false 2023-06-24T09:28:13.062 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-06-24T09:28:13.062 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-06-24T09:28:13.062 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-06-24T09:28:13.062 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.blob_file_starting_level: 0 2023-06-24T09:28:13.062 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-06-24T09:28:13.062 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi188/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-06-24T09:28:13.063 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-06-24T09:28:13.063 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 61f61d6b-bc76-4e66-a300-2f03c13a91a3 2023-06-24T09:28:13.063 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687598892754020, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-06-24T09:28:13.063 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-06-24T09:28:13.064 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687598892754551, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1687598892, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "61f61d6b-bc76-4e66-a300-2f03c13a91a3", "db_session_id": "DQ4HZW349XSZBIBH3JMS", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-06-24T09:28:13.064 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687598892754648, "job": 1, "event": "recovery_finished"} 2023-06-24T09:28:13.064 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-06-24T09:28:13.064 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 systemd[1]: Started Ceph mon.smithi188 for fceaf654-1270-11ee-9b2b-001a4aab830c. 2023-06-24T09:28:13.065 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi188/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-06-24T09:28:13.065 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5650bb738000 2023-06-24T09:28:13.065 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: DB pointer 0x5650bb724000 2023-06-24T09:28:13.065 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-06-24T09:28:13.066 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-06-24T09:28:13.066 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: ** DB Stats ** 2023-06-24T09:28:13.066 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-24T09:28:13.066 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-06-24T09:28:13.066 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-24T09:28:13.067 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-24T09:28:13.067 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-06-24T09:28:13.067 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-24T09:28:13.067 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-24T09:28:13.067 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: 2023-06-24T09:28:13.068 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: ** Compaction Stats [default] ** 2023-06-24T09:28:13.068 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-06-24T09:28:13.068 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-06-24T09:28:13.068 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-24T09:28:13.069 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-24T09:28:13.069 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-24T09:28:13.069 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: 2023-06-24T09:28:13.069 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: ** Compaction Stats [default] ** 2023-06-24T09:28:13.069 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-06-24T09:28:13.070 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-24T09:28:13.070 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-24T09:28:13.070 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: 2023-06-24T09:28:13.070 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-06-24T09:28:13.071 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: 2023-06-24T09:28:13.071 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-24T09:28:13.071 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-24T09:28:13.071 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-24T09:28:13.072 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-24T09:28:13.072 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-24T09:28:13.072 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-24T09:28:13.073 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Cumulative compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-24T09:28:13.073 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Interval compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-24T09:28:13.073 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-24T09:28:13.073 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Block cache BinnedLRUCache@0x5650bb69b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2023-06-24T09:28:13.073 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-06-24T09:28:13.073 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: 2023-06-24T09:28:13.073 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-24T09:28:13.074 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188 does not exist in monmap, will attempt to join an existing cluster 2023-06-24T09:28:13.074 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: using public_addr v2:172.21.15.188:0/0 -> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2023-06-24T09:28:13.074 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: starting mon.smithi188 rank -1 at public addrs [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] at bind addrs [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi188 fsid fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:28:13.074 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(???) e0 preinit fsid fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:28:13.074 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).mds e1 new map 2023-06-24T09:28:13.074 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).mds e1 print_map 2023-06-24T09:28:13.074 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: e1 2023-06-24T09:28:13.075 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-06-24T09:28:13.075 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-06-24T09:28:13.075 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: legacy client fscid: -1 2023-06-24T09:28:13.075 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: 2023-06-24T09:28:13.075 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout: No filesystems configured 2023-06-24T09:28:13.075 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-06-24T09:28:13.075 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-06-24T09:28:13.075 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-06-24T09:28:13.076 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-06-24T09:28:13.076 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-06-24T09:28:13.076 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-06-24T09:28:13.076 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2023-06-24T09:28:13.076 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2023-06-24T09:28:13.076 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2023-06-24T09:28:13.076 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2023-06-24T09:28:13.077 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2023-06-24T09:28:13.077 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: Deploying daemon node-exporter.smithi188 on smithi188 2023-06-24T09:28:13.077 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/2389529030' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:28:13.077 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:13.077 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:13.077 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:13.077 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:13.077 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi188.zipfzj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-24T09:28:13.078 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi188.zipfzj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-24T09:28:13.078 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-24T09:28:13.078 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:13.078 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: Deploying daemon mgr.smithi188.zipfzj on smithi188 2023-06-24T09:28:13.078 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:28:13.078 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:13.079 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/798883289' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:28:13.079 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:13.079 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:13.080 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:13.080 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:13.080 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-24T09:28:13.081 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:13.081 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: Deploying daemon mon.smithi188 on smithi188 2023-06-24T09:28:13.081 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:12 smithi188 ceph-mon[112326]: mon.smithi188@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2023-06-24T09:28:17.845 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-06-24T09:28:17.845 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":2,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","modified":"2023-06-24T09:28:12.791196Z","created":"2023-06-24T09:25:34.820744Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi045","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi188","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-06-24T09:28:17.847 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 2 2023-06-24T09:28:18.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi045"}]: dispatch 2023-06-24T09:28:18.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: mon.smithi045 calling monitor election 2023-06-24T09:28:18.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/2104881033' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:28:18.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:18.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: mon.smithi188 calling monitor election 2023-06-24T09:28:18.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:18.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.? 172.21.15.188:0/627093594' entity='mgr.smithi188.zipfzj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi188.zipfzj/crt"}]: dispatch 2023-06-24T09:28:18.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:18.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: mon.smithi045 is new leader, mons smithi045,smithi188 in quorum (ranks 0,1) 2023-06-24T09:28:18.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: monmap e2: 2 mons at {smithi045=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],smithi188=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} 2023-06-24T09:28:18.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: fsmap 2023-06-24T09:28:18.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: osdmap e5: 0 total, 0 up, 0 in 2023-06-24T09:28:18.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: mgrmap e17: smithi045.dcgntd(active, since 24s) 2023-06-24T09:28:18.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: Standby manager daemon smithi188.zipfzj started 2023-06-24T09:28:18.136 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.? 172.21.15.188:0/627093594' entity='mgr.smithi188.zipfzj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-24T09:28:18.136 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: overall HEALTH_OK 2023-06-24T09:28:18.136 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.? 172.21.15.188:0/627093594' entity='mgr.smithi188.zipfzj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi188.zipfzj/key"}]: dispatch 2023-06-24T09:28:18.136 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.? 172.21.15.188:0/627093594' entity='mgr.smithi188.zipfzj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-24T09:28:18.136 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:18.136 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:18.136 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:18.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi045"}]: dispatch 2023-06-24T09:28:18.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: mon.smithi045 calling monitor election 2023-06-24T09:28:18.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/2104881033' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-24T09:28:18.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:18.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: mon.smithi188 calling monitor election 2023-06-24T09:28:18.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:18.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.? 172.21.15.188:0/627093594' entity='mgr.smithi188.zipfzj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi188.zipfzj/crt"}]: dispatch 2023-06-24T09:28:18.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:18.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:18.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: mon.smithi045 is new leader, mons smithi045,smithi188 in quorum (ranks 0,1) 2023-06-24T09:28:18.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: monmap e2: 2 mons at {smithi045=[v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0],smithi188=[v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0]} removed_ranks: {} 2023-06-24T09:28:18.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: fsmap 2023-06-24T09:28:18.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: osdmap e5: 0 total, 0 up, 0 in 2023-06-24T09:28:18.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: mgrmap e17: smithi045.dcgntd(active, since 24s) 2023-06-24T09:28:18.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: Standby manager daemon smithi188.zipfzj started 2023-06-24T09:28:18.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.? 172.21.15.188:0/627093594' entity='mgr.smithi188.zipfzj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-24T09:28:18.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: overall HEALTH_OK 2023-06-24T09:28:18.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.? 172.21.15.188:0/627093594' entity='mgr.smithi188.zipfzj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi188.zipfzj/key"}]: dispatch 2023-06-24T09:28:18.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.? 172.21.15.188:0/627093594' entity='mgr.smithi188.zipfzj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-24T09:28:18.175 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:18.175 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:18.175 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:18.274 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-06-24T09:28:18.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph config generate-minimal-conf 2023-06-24T09:28:18.526 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:19.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:18 smithi045 ceph-mon[102509]: mgrmap e18: smithi045.dcgntd(active, since 24s), standbys: smithi188.zipfzj 2023-06-24T09:28:19.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr metadata", "who": "smithi188.zipfzj", "id": "smithi188.zipfzj"}]: dispatch 2023-06-24T09:28:19.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:19.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:19.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:19.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:28:19.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:19.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:18 smithi188 ceph-mon[112326]: mgrmap e18: smithi045.dcgntd(active, since 24s), standbys: smithi188.zipfzj 2023-06-24T09:28:19.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr metadata", "who": "smithi188.zipfzj", "id": "smithi188.zipfzj"}]: dispatch 2023-06-24T09:28:19.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:19.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:19.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:19.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:28:19.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:28:19.927 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:28:19.928 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2023-06-24T09:28:19.928 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = fceaf654-1270-11ee-9b2b-001a4aab830c 2023-06-24T09:28:19.928 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2023-06-24T09:28:20.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:19 smithi045 ceph-mon[102509]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:20.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:20.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:28:20.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.332 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-06-24T09:28:20.332 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:28:20.333 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2023-06-24T09:28:20.363 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:28:20.363 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-24T09:28:20.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:19 smithi188 ceph-mon[112326]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:20.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:20.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:28:20.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:20.431 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-06-24T09:28:20.432 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.conf 2023-06-24T09:28:20.466 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-06-24T09:28:20.466 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-24T09:28:20.534 INFO:tasks.cephadm:Deploying OSDs... 2023-06-24T09:28:20.535 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:28:20.535 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2023-06-24T09:28:20.553 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2023-06-24T09:28:20.554 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/nvme1n1 2023-06-24T09:28:20.612 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/nvme1n1 2023-06-24T09:28:20.613 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-06-24T09:28:20.613 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 407446 Links: 1 Device type: 103,1 2023-06-24T09:28:20.613 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-06-24T09:28:20.613 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:nvme_device_t:s0 2023-06-24T09:28:20.613 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-06-24 09:26:42.327109095 +0000 2023-06-24T09:28:20.614 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-06-24 09:26:42.264110038 +0000 2023-06-24T09:28:20.614 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-06-24 09:26:42.264110038 +0000 2023-06-24T09:28:20.614 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-06-24T09:28:20.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2023-06-24T09:28:20.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-06-24T09:28:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-06-24T09:28:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000121005 s, 4.2 MB/s 2023-06-24T09:28:20.682 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2023-06-24T09:28:20.741 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/nvme2n1 2023-06-24T09:28:20.800 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/nvme2n1 2023-06-24T09:28:20.800 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-06-24T09:28:20.800 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 407523 Links: 1 Device type: 103,2 2023-06-24T09:28:20.800 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-06-24T09:28:20.800 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:nvme_device_t:s0 2023-06-24T09:28:20.800 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-06-24 09:26:42.336108961 +0000 2023-06-24T09:28:20.801 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-06-24 09:26:42.265110023 +0000 2023-06-24T09:28:20.801 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-06-24 09:26:42.265110023 +0000 2023-06-24T09:28:20.801 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-06-24T09:28:20.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2023-06-24T09:28:20.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-06-24T09:28:20.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-06-24T09:28:20.867 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000118825 s, 4.3 MB/s 2023-06-24T09:28:20.868 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2023-06-24T09:28:20.927 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/nvme3n1 2023-06-24T09:28:20.985 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/nvme3n1 2023-06-24T09:28:20.985 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-06-24T09:28:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 413191 Links: 1 Device type: 103,3 2023-06-24T09:28:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-06-24T09:28:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:nvme_device_t:s0 2023-06-24T09:28:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-06-24 09:26:42.345108826 +0000 2023-06-24T09:28:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-06-24 09:26:42.269109963 +0000 2023-06-24T09:28:20.986 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-06-24 09:26:42.269109963 +0000 2023-06-24T09:28:20.986 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-06-24T09:28:20.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2023-06-24T09:28:21.050 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:20 smithi045 ceph-mon[102509]: Updating smithi045:/etc/ceph/ceph.conf 2023-06-24T09:28:21.051 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:20 smithi045 ceph-mon[102509]: Updating smithi188:/etc/ceph/ceph.conf 2023-06-24T09:28:21.051 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:20 smithi045 ceph-mon[102509]: Updating smithi045:/var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-24T09:28:21.051 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:20 smithi045 ceph-mon[102509]: Updating smithi188:/var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-24T09:28:21.051 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:20 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1374332125' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:21.051 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:20 smithi045 ceph-mon[102509]: Reconfiguring mon.smithi045 (unknown last config time)... 2023-06-24T09:28:21.051 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-24T09:28:21.051 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-24T09:28:21.052 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:21.052 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:20 smithi045 ceph-mon[102509]: Reconfiguring daemon mon.smithi045 on smithi045 2023-06-24T09:28:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-06-24T09:28:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-06-24T09:28:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000106967 s, 4.8 MB/s 2023-06-24T09:28:21.053 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2023-06-24T09:28:21.113 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/nvme4n1 2023-06-24T09:28:21.172 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/nvme4n1 2023-06-24T09:28:21.172 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-06-24T09:28:21.173 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 413290 Links: 1 Device type: 103,4 2023-06-24T09:28:21.173 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-06-24T09:28:21.173 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:nvme_device_t:s0 2023-06-24T09:28:21.173 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2023-06-24 09:26:42.354108692 +0000 2023-06-24T09:28:21.173 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2023-06-24 09:26:42.271109933 +0000 2023-06-24T09:28:21.173 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2023-06-24 09:26:42.271109933 +0000 2023-06-24T09:28:21.174 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: - 2023-06-24T09:28:21.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2023-06-24T09:28:21.194 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:20 smithi188 ceph-mon[112326]: Updating smithi045:/etc/ceph/ceph.conf 2023-06-24T09:28:21.194 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:20 smithi188 ceph-mon[112326]: Updating smithi188:/etc/ceph/ceph.conf 2023-06-24T09:28:21.194 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:20 smithi188 ceph-mon[112326]: Updating smithi045:/var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-24T09:28:21.194 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:20 smithi188 ceph-mon[112326]: Updating smithi188:/var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-24T09:28:21.194 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:20 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1374332125' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:21.194 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:20 smithi188 ceph-mon[112326]: Reconfiguring mon.smithi045 (unknown last config time)... 2023-06-24T09:28:21.195 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-24T09:28:21.195 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-24T09:28:21.195 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:21.195 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:20 smithi188 ceph-mon[112326]: Reconfiguring daemon mon.smithi045 on smithi045 2023-06-24T09:28:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2023-06-24T09:28:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2023-06-24T09:28:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000122413 s, 4.2 MB/s 2023-06-24T09:28:21.241 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2023-06-24T09:28:21.300 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-06-24T09:28:21.300 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2023-06-24T09:28:21.322 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2023-06-24T09:28:21.323 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/nvme1n1 2023-06-24T09:28:21.383 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/nvme1n1 2023-06-24T09:28:21.384 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-06-24T09:28:21.384 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 410344 Links: 1 Device type: 103,1 2023-06-24T09:28:21.384 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-06-24T09:28:21.384 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:nvme_device_t:s0 2023-06-24T09:28:21.384 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-06-24 09:27:56.628461926 +0000 2023-06-24T09:28:21.385 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-06-24 09:27:56.579462784 +0000 2023-06-24T09:28:21.385 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-06-24 09:27:56.579462784 +0000 2023-06-24T09:28:21.385 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-06-24T09:28:21.385 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2023-06-24T09:28:21.454 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-06-24T09:28:21.455 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-06-24T09:28:21.455 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000299543 s, 1.7 MB/s 2023-06-24T09:28:21.456 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2023-06-24T09:28:21.517 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/nvme2n1 2023-06-24T09:28:21.577 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/nvme2n1 2023-06-24T09:28:21.577 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-06-24T09:28:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 410386 Links: 1 Device type: 103,2 2023-06-24T09:28:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-06-24T09:28:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:nvme_device_t:s0 2023-06-24T09:28:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-06-24 09:27:56.640461716 +0000 2023-06-24T09:28:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-06-24 09:27:56.581462749 +0000 2023-06-24T09:28:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-06-24 09:27:56.581462749 +0000 2023-06-24T09:28:21.578 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-06-24T09:28:21.578 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2023-06-24T09:28:21.646 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-06-24T09:28:21.646 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-06-24T09:28:21.646 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000255703 s, 2.0 MB/s 2023-06-24T09:28:21.648 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2023-06-24T09:28:21.709 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/nvme3n1 2023-06-24T09:28:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/nvme3n1 2023-06-24T09:28:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-06-24T09:28:21.769 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 409237 Links: 1 Device type: 103,3 2023-06-24T09:28:21.769 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-06-24T09:28:21.769 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:nvme_device_t:s0 2023-06-24T09:28:21.769 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-06-24 09:27:56.651461523 +0000 2023-06-24T09:28:21.769 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-06-24 09:27:56.584462696 +0000 2023-06-24T09:28:21.770 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-06-24 09:27:56.584462696 +0000 2023-06-24T09:28:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-06-24T09:28:21.770 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2023-06-24T09:28:21.837 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-06-24T09:28:21.838 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-06-24T09:28:21.838 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000223605 s, 2.3 MB/s 2023-06-24T09:28:21.839 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2023-06-24T09:28:21.899 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/nvme4n1 2023-06-24T09:28:21.958 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/nvme4n1 2023-06-24T09:28:21.958 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2023-06-24T09:28:21.959 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 6h/6d Inode: 409247 Links: 1 Device type: 103,4 2023-06-24T09:28:21.959 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2023-06-24T09:28:21.959 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:nvme_device_t:s0 2023-06-24T09:28:21.959 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-06-24 09:27:56.662461330 +0000 2023-06-24T09:28:21.959 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-06-24 09:27:56.588462626 +0000 2023-06-24T09:28:21.960 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-06-24 09:27:56.588462626 +0000 2023-06-24T09:28:21.960 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: - 2023-06-24T09:28:21.960 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2023-06-24T09:28:22.025 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-06-24T09:28:22.025 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-06-24T09:28:22.025 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.00026064 s, 2.0 MB/s 2023-06-24T09:28:22.026 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2023-06-24T09:28:22.086 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch apply osd --all-available-devices 2023-06-24T09:28:22.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:22 smithi045 ceph-mon[102509]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:22.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:22.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:22.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:22 smithi045 ceph-mon[102509]: Reconfiguring mgr.smithi045.dcgntd (unknown last config time)... 2023-06-24T09:28:22.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi045.dcgntd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-24T09:28:22.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-24T09:28:22.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:22.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:22 smithi045 ceph-mon[102509]: Reconfiguring daemon mgr.smithi045.dcgntd on smithi045 2023-06-24T09:28:22.345 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi188/config 2023-06-24T09:28:22.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:22 smithi188 ceph-mon[112326]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:22.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:22.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:22.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:22 smithi188 ceph-mon[112326]: Reconfiguring mgr.smithi045.dcgntd (unknown last config time)... 2023-06-24T09:28:22.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi045.dcgntd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-24T09:28:22.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-24T09:28:22.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:22.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:22 smithi188 ceph-mon[112326]: Reconfiguring daemon mgr.smithi045.dcgntd on smithi045 2023-06-24T09:28:23.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:23.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:23.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi045", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2023-06-24T09:28:23.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:23.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:23.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:28:23.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:23.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:23.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi045", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2023-06-24T09:28:23.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:23.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:23.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:28:23.736 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled osd.all-available-devices update... 2023-06-24T09:28:24.124 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2023-06-24T09:28:24.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:28:24.376 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:24.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:24 smithi188 ceph-mon[112326]: Reconfiguring ceph-exporter.smithi045 (monmap changed)... 2023-06-24T09:28:24.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:24 smithi188 ceph-mon[112326]: Reconfiguring daemon ceph-exporter.smithi045 on smithi045 2023-06-24T09:28:24.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:24 smithi188 ceph-mon[112326]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:24.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:24.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:24.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi045", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2023-06-24T09:28:24.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:24.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:24.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:24 smithi045 ceph-mon[102509]: Reconfiguring ceph-exporter.smithi045 (monmap changed)... 2023-06-24T09:28:24.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:24 smithi045 ceph-mon[102509]: Reconfiguring daemon ceph-exporter.smithi045 on smithi045 2023-06-24T09:28:24.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:24 smithi045 ceph-mon[102509]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:24.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:24.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:24.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi045", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2023-06-24T09:28:24.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:24.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:25.234 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:25 smithi045 ceph-mon[102509]: Reconfiguring crash.smithi045 (monmap changed)... 2023-06-24T09:28:25.235 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:25 smithi045 ceph-mon[102509]: Reconfiguring daemon crash.smithi045 on smithi045 2023-06-24T09:28:25.235 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:25 smithi045 ceph-mon[102509]: from='client.24099 -' entity='client.admin' cmd=[{"prefix": "orch apply osd", "all_available_devices": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:28:25.235 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:25 smithi045 ceph-mon[102509]: Marking host: smithi045 for OSDSpec preview refresh. 2023-06-24T09:28:25.235 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:25 smithi045 ceph-mon[102509]: Marking host: smithi188 for OSDSpec preview refresh. 2023-06-24T09:28:25.235 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:25 smithi045 ceph-mon[102509]: Saving service osd.all-available-devices spec with placement * 2023-06-24T09:28:25.235 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:25 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:25.236 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:25 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:25.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:25 smithi188 ceph-mon[112326]: Reconfiguring crash.smithi045 (monmap changed)... 2023-06-24T09:28:25.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:25 smithi188 ceph-mon[112326]: Reconfiguring daemon crash.smithi045 on smithi045 2023-06-24T09:28:25.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:25 smithi188 ceph-mon[112326]: from='client.24099 -' entity='client.admin' cmd=[{"prefix": "orch apply osd", "all_available_devices": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:28:25.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:25 smithi188 ceph-mon[112326]: Marking host: smithi045 for OSDSpec preview refresh. 2023-06-24T09:28:25.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:25 smithi188 ceph-mon[112326]: Marking host: smithi188 for OSDSpec preview refresh. 2023-06-24T09:28:25.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:25 smithi188 ceph-mon[112326]: Saving service osd.all-available-devices spec with placement * 2023-06-24T09:28:25.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:25 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:25.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:25 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:28:26.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:26 smithi188 ceph-mon[112326]: Reconfiguring alertmanager.smithi045 (dependencies changed)... 2023-06-24T09:28:26.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:26 smithi188 ceph-mon[112326]: Reconfiguring daemon alertmanager.smithi045 on smithi045 2023-06-24T09:28:26.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:26 smithi188 ceph-mon[112326]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:26.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:26 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/2948752653' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:26.409 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:26 smithi045 ceph-mon[102509]: Reconfiguring alertmanager.smithi045 (dependencies changed)... 2023-06-24T09:28:26.409 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:26 smithi045 ceph-mon[102509]: Reconfiguring daemon alertmanager.smithi045 on smithi045 2023-06-24T09:28:26.409 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:26 smithi045 ceph-mon[102509]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:26.409 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:26 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2948752653' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:26.909 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0} 2023-06-24T09:28:27.911 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:28:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:28.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:28 smithi188 ceph-mon[112326]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:28.413 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:28 smithi045 ceph-mon[102509]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:29.932 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:28:30.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:30 smithi188 ceph-mon[112326]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:30.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:30 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:30.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:30 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:30.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:30 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/579081208' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:30.399 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:30 smithi045 ceph-mon[102509]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:30.399 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:30 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:30.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:30 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:30.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:30 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/579081208' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:30.487 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0} 2023-06-24T09:28:31.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:31 smithi188 ceph-mon[112326]: Reconfiguring grafana.smithi045 (dependencies changed)... 2023-06-24T09:28:31.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:31 smithi188 ceph-mon[112326]: Reconfiguring daemon grafana.smithi045 on smithi045 2023-06-24T09:28:31.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:31 smithi045 ceph-mon[102509]: Reconfiguring grafana.smithi045 (dependencies changed)... 2023-06-24T09:28:31.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:31 smithi045 ceph-mon[102509]: Reconfiguring daemon grafana.smithi045 on smithi045 2023-06-24T09:28:31.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:28:31.759 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:32.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:32 smithi045 ceph-mon[102509]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:32.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:32 smithi188 ceph-mon[112326]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:33.263 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:33 smithi045 ceph-mon[102509]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:33.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:33 smithi188 ceph-mon[112326]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:33.656 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:28:34.109 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0} 2023-06-24T09:28:34.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:34 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3436407281' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:34.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:34 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:34.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:34 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:34.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:34 smithi045 ceph-mon[102509]: Reconfiguring prometheus.smithi045 (dependencies changed)... 2023-06-24T09:28:34.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:34 smithi045 ceph-mon[102509]: Reconfiguring daemon prometheus.smithi045 on smithi045 2023-06-24T09:28:34.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:34 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3436407281' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:34.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:34 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:34.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:34 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:34.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:34 smithi188 ceph-mon[112326]: Reconfiguring prometheus.smithi045 (dependencies changed)... 2023-06-24T09:28:34.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:34 smithi188 ceph-mon[112326]: Reconfiguring daemon prometheus.smithi045 on smithi045 2023-06-24T09:28:35.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:28:35.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:35 smithi045 ceph-mon[102509]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:35.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:35 smithi188 ceph-mon[112326]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:37.286 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:28:37.751 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0} 2023-06-24T09:28:38.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:38 smithi188 ceph-mon[112326]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:38.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:38 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3620831347' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:38.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:28:38.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:38 smithi045 ceph-mon[102509]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:38.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:38 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3620831347' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:38.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:28:38.752 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:28:38.997 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:39.560 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:39.560 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:39.560 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:39 smithi188 ceph-mon[112326]: Reconfiguring ceph-exporter.smithi188 (monmap changed)... 2023-06-24T09:28:39.561 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi188", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2023-06-24T09:28:39.561 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:39.561 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:39 smithi188 ceph-mon[112326]: Reconfiguring daemon ceph-exporter.smithi188 on smithi188 2023-06-24T09:28:39.561 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:39 smithi188 ceph-mon[112326]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:39.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:39.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:39.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:39 smithi045 ceph-mon[102509]: Reconfiguring ceph-exporter.smithi188 (monmap changed)... 2023-06-24T09:28:39.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi188", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2023-06-24T09:28:39.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:39.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:39 smithi045 ceph-mon[102509]: Reconfiguring daemon ceph-exporter.smithi188 on smithi188 2023-06-24T09:28:39.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:39 smithi045 ceph-mon[102509]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:28:40.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:40 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:40.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:40 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:40.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:40 smithi045 ceph-mon[102509]: Reconfiguring crash.smithi188 (monmap changed)... 2023-06-24T09:28:40.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:40 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi188", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2023-06-24T09:28:40.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:40 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:40.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:40 smithi045 ceph-mon[102509]: Reconfiguring daemon crash.smithi188 on smithi188 2023-06-24T09:28:40.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:40 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3692571815' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:40.754 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0} 2023-06-24T09:28:40.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:40 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:40.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:40 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:40.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:40 smithi188 ceph-mon[112326]: Reconfiguring crash.smithi188 (monmap changed)... 2023-06-24T09:28:40.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:40 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi188", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2023-06-24T09:28:40.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:40 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:40.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:40 smithi188 ceph-mon[112326]: Reconfiguring daemon crash.smithi188 on smithi188 2023-06-24T09:28:40.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:40 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3692571815' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:41.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:28:41.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:41 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:41.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:41 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:41.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:41 smithi045 ceph-mon[102509]: Reconfiguring mgr.smithi188.zipfzj (monmap changed)... 2023-06-24T09:28:41.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:41 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi188.zipfzj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-24T09:28:41.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:41 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-24T09:28:41.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:41 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:41.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:41 smithi045 ceph-mon[102509]: Reconfiguring daemon mgr.smithi188.zipfzj on smithi188 2023-06-24T09:28:41.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:41 smithi045 ceph-mon[102509]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:42.008 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:42.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:41 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:42.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:41 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:42.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:41 smithi188 ceph-mon[112326]: Reconfiguring mgr.smithi188.zipfzj (monmap changed)... 2023-06-24T09:28:42.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:41 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi188.zipfzj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-24T09:28:42.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:41 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-24T09:28:42.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:41 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:42.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:41 smithi188 ceph-mon[112326]: Reconfiguring daemon mgr.smithi188.zipfzj on smithi188 2023-06-24T09:28:42.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:41 smithi188 ceph-mon[112326]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:42.985 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:42.986 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:42.986 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:42 smithi045 ceph-mon[102509]: Reconfiguring mon.smithi188 (monmap changed)... 2023-06-24T09:28:42.986 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-24T09:28:42.986 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-24T09:28:42.986 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:42.986 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:42 smithi045 ceph-mon[102509]: Reconfiguring daemon mon.smithi188 on smithi188 2023-06-24T09:28:43.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:43.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:43.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:42 smithi188 ceph-mon[112326]: Reconfiguring mon.smithi188 (monmap changed)... 2023-06-24T09:28:43.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-24T09:28:43.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-24T09:28:43.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:43.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:42 smithi188 ceph-mon[112326]: Reconfiguring daemon mon.smithi188 on smithi188 2023-06-24T09:28:43.452 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:28:43.943 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0} 2023-06-24T09:28:44.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:44.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:44.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-06-24T09:28:44.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-06-24T09:28:44.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi045.front.sepia.ceph.com:9093"}]: dispatch 2023-06-24T09:28:44.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi045.front.sepia.ceph.com:9093"}]: dispatch 2023-06-24T09:28:44.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-06-24T09:28:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-06-24T09:28:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi045.front.sepia.ceph.com:3000"}]: dispatch 2023-06-24T09:28:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi045.front.sepia.ceph.com:3000"}]: dispatch 2023-06-24T09:28:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-06-24T09:28:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-06-24T09:28:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi045.front.sepia.ceph.com:9095"}]: dispatch 2023-06-24T09:28:44.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi045.front.sepia.ceph.com:9095"}]: dispatch 2023-06-24T09:28:44.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:44.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:28:44.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:44.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:43 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3027003396' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:44.170 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-06-24T09:28:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2023-06-24T09:28:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi045.front.sepia.ceph.com:9093"}]: dispatch 2023-06-24T09:28:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi045.front.sepia.ceph.com:9093"}]: dispatch 2023-06-24T09:28:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-06-24T09:28:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2023-06-24T09:28:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi045.front.sepia.ceph.com:3000"}]: dispatch 2023-06-24T09:28:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi045.front.sepia.ceph.com:3000"}]: dispatch 2023-06-24T09:28:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-06-24T09:28:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2023-06-24T09:28:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi045.front.sepia.ceph.com:9095"}]: dispatch 2023-06-24T09:28:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi045.front.sepia.ceph.com:9095"}]: dispatch 2023-06-24T09:28:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:28:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:43 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3027003396' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:44.944 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:28:45.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:45.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:45.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:45.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:45.346 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:46.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:46.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:46.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:46.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:46.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:46.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:46.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:28:46.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:46.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:28:46.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-24T09:28:46.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:46.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-24T09:28:46.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:46.226 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:46.227 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:46.227 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:46.227 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:46.227 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:46.227 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:46.227 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:28:46.227 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:28:46.228 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:28:46.228 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-24T09:28:46.228 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:46.228 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-24T09:28:46.228 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:28:47.384 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:28:47.806 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0} 2023-06-24T09:28:48.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:48 smithi188 ceph-mon[112326]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:48.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:48 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1422697437' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:48.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:48 smithi045 ceph-mon[102509]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:48.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:48 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1422697437' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:48.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:28:49.052 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:49.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:49 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/2132342985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "602c763e-3002-4a3a-8243-abf07f5096b6"}]: dispatch 2023-06-24T09:28:49.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:49 smithi045 ceph-mon[102509]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "602c763e-3002-4a3a-8243-abf07f5096b6"}]: dispatch 2023-06-24T09:28:49.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:49 smithi045 ceph-mon[102509]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "602c763e-3002-4a3a-8243-abf07f5096b6"}]': finished 2023-06-24T09:28:49.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:49 smithi045 ceph-mon[102509]: osdmap e6: 1 total, 0 up, 1 in 2023-06-24T09:28:49.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:49 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:49.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:49 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/2132342985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "602c763e-3002-4a3a-8243-abf07f5096b6"}]: dispatch 2023-06-24T09:28:49.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:49 smithi188 ceph-mon[112326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "602c763e-3002-4a3a-8243-abf07f5096b6"}]: dispatch 2023-06-24T09:28:49.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:49 smithi188 ceph-mon[112326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "602c763e-3002-4a3a-8243-abf07f5096b6"}]': finished 2023-06-24T09:28:49.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:49 smithi188 ceph-mon[112326]: osdmap e6: 1 total, 0 up, 1 in 2023-06-24T09:28:49.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:49 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:50.285 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:50 smithi045 ceph-mon[102509]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:50.285 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:50 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/220007057' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c41d63a-21f0-429d-8ca5-0d07abdb8b0e"}]: dispatch 2023-06-24T09:28:50.286 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:50 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/220007057' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c41d63a-21f0-429d-8ca5-0d07abdb8b0e"}]': finished 2023-06-24T09:28:50.286 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:50 smithi045 ceph-mon[102509]: osdmap e7: 2 total, 0 up, 2 in 2023-06-24T09:28:50.286 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:50 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:50.286 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:50 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:28:50.286 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:50 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/3859119691' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:28:50.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:50 smithi188 ceph-mon[112326]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:50.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:50 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/220007057' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c41d63a-21f0-429d-8ca5-0d07abdb8b0e"}]: dispatch 2023-06-24T09:28:50.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:50 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/220007057' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c41d63a-21f0-429d-8ca5-0d07abdb8b0e"}]': finished 2023-06-24T09:28:50.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:50 smithi188 ceph-mon[112326]: osdmap e7: 2 total, 0 up, 2 in 2023-06-24T09:28:50.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:50 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:50.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:50 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:28:50.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:50 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/3859119691' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:28:50.721 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:28:51.149 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":7,"num_osds":2,"num_up_osds":0,"osd_up_since":0,"num_in_osds":2,"osd_in_since":1687598929,"num_remapped_pgs":0} 2023-06-24T09:28:51.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:51 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3103435801' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:51.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:51 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/361758103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:28:51.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:51 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3103435801' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:51.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:51 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/361758103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:28:52.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:28:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:52 smithi188 ceph-mon[112326]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:52 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/3095955151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313dc3fd-6227-4d95-a488-2f9d1368aa6a"}]: dispatch 2023-06-24T09:28:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:52 smithi188 ceph-mon[112326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313dc3fd-6227-4d95-a488-2f9d1368aa6a"}]: dispatch 2023-06-24T09:28:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:52 smithi188 ceph-mon[112326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "313dc3fd-6227-4d95-a488-2f9d1368aa6a"}]': finished 2023-06-24T09:28:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:52 smithi188 ceph-mon[112326]: osdmap e8: 3 total, 0 up, 3 in 2023-06-24T09:28:52.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:52.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:28:52.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:28:52.403 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:52.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:52 smithi045 ceph-mon[102509]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:52.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:52 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/3095955151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313dc3fd-6227-4d95-a488-2f9d1368aa6a"}]: dispatch 2023-06-24T09:28:52.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:52 smithi045 ceph-mon[102509]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "313dc3fd-6227-4d95-a488-2f9d1368aa6a"}]: dispatch 2023-06-24T09:28:52.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:52 smithi045 ceph-mon[102509]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "313dc3fd-6227-4d95-a488-2f9d1368aa6a"}]': finished 2023-06-24T09:28:52.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:52 smithi045 ceph-mon[102509]: osdmap e8: 3 total, 0 up, 3 in 2023-06-24T09:28:52.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:52.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:28:52.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:28:53.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:28:53.392 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:28:53.827 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:28:54.118 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:54 smithi045 ceph-mon[102509]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:54.119 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:54 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/2104613897' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:28:54.119 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:54 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/4121089674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c1a9355-00bc-4add-bac9-249ed07d0c94"}]: dispatch 2023-06-24T09:28:54.119 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:54 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/4121089674' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c1a9355-00bc-4add-bac9-249ed07d0c94"}]': finished 2023-06-24T09:28:54.119 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:54 smithi045 ceph-mon[102509]: osdmap e9: 4 total, 0 up, 4 in 2023-06-24T09:28:54.119 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:54 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:54.119 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:54 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:28:54.119 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:54 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:28:54.120 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:54 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:28:54.120 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:54 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1388953647' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:54 smithi188 ceph-mon[112326]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:54.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:54 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/2104613897' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:28:54.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:54 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/4121089674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c1a9355-00bc-4add-bac9-249ed07d0c94"}]: dispatch 2023-06-24T09:28:54.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:54 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/4121089674' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c1a9355-00bc-4add-bac9-249ed07d0c94"}]': finished 2023-06-24T09:28:54.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:54 smithi188 ceph-mon[112326]: osdmap e9: 4 total, 0 up, 4 in 2023-06-24T09:28:54.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:54 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:54.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:54 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:28:54.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:54 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:28:54.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:54 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:28:54.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:54 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1388953647' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"num_osds":4,"num_up_osds":0,"osd_up_since":0,"num_in_osds":4,"osd_in_since":1687598933,"num_remapped_pgs":0} 2023-06-24T09:28:55.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:55 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3625229914' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:28:55.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:55 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3625229914' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:28:55.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:28:55.751 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:56.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:56 smithi188 ceph-mon[112326]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:56.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:56 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/2699920004' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b392039a-ca73-47b7-9164-a9918a4c10f5"}]: dispatch 2023-06-24T09:28:56.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:56 smithi188 ceph-mon[112326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b392039a-ca73-47b7-9164-a9918a4c10f5"}]: dispatch 2023-06-24T09:28:56.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:56 smithi188 ceph-mon[112326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b392039a-ca73-47b7-9164-a9918a4c10f5"}]': finished 2023-06-24T09:28:56.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:56 smithi188 ceph-mon[112326]: osdmap e10: 5 total, 0 up, 5 in 2023-06-24T09:28:56.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:56 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:56.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:56 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:28:56.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:56 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:28:56.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:56 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:28:56.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:56 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:28:56.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:56 smithi045 ceph-mon[102509]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:56.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:56 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/2699920004' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b392039a-ca73-47b7-9164-a9918a4c10f5"}]: dispatch 2023-06-24T09:28:56.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:56 smithi045 ceph-mon[102509]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b392039a-ca73-47b7-9164-a9918a4c10f5"}]: dispatch 2023-06-24T09:28:56.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:56 smithi045 ceph-mon[102509]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b392039a-ca73-47b7-9164-a9918a4c10f5"}]': finished 2023-06-24T09:28:56.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:56 smithi045 ceph-mon[102509]: osdmap e10: 5 total, 0 up, 5 in 2023-06-24T09:28:56.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:56 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:56.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:56 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:28:56.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:56 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:28:56.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:56 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:28:56.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:56 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:28:57.213 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:28:57.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:57 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/3961332771' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:28:57.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:57 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/3961332771' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:28:57.643 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"num_osds":5,"num_up_osds":0,"osd_up_since":0,"num_in_osds":5,"osd_in_since":1687598935,"num_remapped_pgs":0} 2023-06-24T09:28:58.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:58 smithi188 ceph-mon[112326]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:58.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:58 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/198143205' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:58.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:58 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1780235844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb292935-34ed-4c37-ad12-969cfee51cba"}]: dispatch 2023-06-24T09:28:58.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:58 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1780235844' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb292935-34ed-4c37-ad12-969cfee51cba"}]': finished 2023-06-24T09:28:58.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:58 smithi188 ceph-mon[112326]: osdmap e11: 6 total, 0 up, 6 in 2023-06-24T09:28:58.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:58 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:58.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:58 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:28:58.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:58 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:28:58.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:58 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:28:58.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:58 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:28:58.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:58 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:28:58.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:58 smithi045 ceph-mon[102509]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:28:58.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:58 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/198143205' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:28:58.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:58 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1780235844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb292935-34ed-4c37-ad12-969cfee51cba"}]: dispatch 2023-06-24T09:28:58.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:58 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1780235844' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb292935-34ed-4c37-ad12-969cfee51cba"}]': finished 2023-06-24T09:28:58.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:58 smithi045 ceph-mon[102509]: osdmap e11: 6 total, 0 up, 6 in 2023-06-24T09:28:58.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:28:58.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:28:58.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:28:58.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:28:58.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:28:58.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:58 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:28:58.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:28:58.891 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:28:59.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:28:59 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1721535036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:28:59.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:28:59 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1721535036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:29:00.239 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:00.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:00.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/275162639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e76afd0b-efa6-45bb-b611-18a5f7db84ac"}]: dispatch 2023-06-24T09:29:00.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e76afd0b-efa6-45bb-b611-18a5f7db84ac"}]: dispatch 2023-06-24T09:29:00.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e76afd0b-efa6-45bb-b611-18a5f7db84ac"}]': finished 2023-06-24T09:29:00.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: osdmap e12: 7 total, 0 up, 7 in 2023-06-24T09:29:00.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:00.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:00.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:00.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:00.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:00.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:00.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:00 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:00.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:00.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/275162639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e76afd0b-efa6-45bb-b611-18a5f7db84ac"}]: dispatch 2023-06-24T09:29:00.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e76afd0b-efa6-45bb-b611-18a5f7db84ac"}]: dispatch 2023-06-24T09:29:00.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e76afd0b-efa6-45bb-b611-18a5f7db84ac"}]': finished 2023-06-24T09:29:00.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: osdmap e12: 7 total, 0 up, 7 in 2023-06-24T09:29:00.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:00.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:00.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:00.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:00.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:00.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:00.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:00 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":12,"num_osds":7,"num_up_osds":0,"osd_up_since":0,"num_in_osds":7,"osd_in_since":1687598939,"num_remapped_pgs":0} 2023-06-24T09:29:01.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:01 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3413132357' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:01.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:01 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/1514454026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:29:01.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:01 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3413132357' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:01.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:01 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/1514454026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:29:01.647 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:29:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:02.369 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:02.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3740783280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4290544c-7c88-4e40-b0b4-8abda55c5c02"}]: dispatch 2023-06-24T09:29:02.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3740783280' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4290544c-7c88-4e40-b0b4-8abda55c5c02"}]': finished 2023-06-24T09:29:02.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: osdmap e13: 8 total, 0 up, 8 in 2023-06-24T09:29:02.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:02.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:02.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:02.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:02.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:02.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:02.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:02.371 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:02 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:02.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:02.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3740783280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4290544c-7c88-4e40-b0b4-8abda55c5c02"}]: dispatch 2023-06-24T09:29:02.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3740783280' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4290544c-7c88-4e40-b0b4-8abda55c5c02"}]': finished 2023-06-24T09:29:02.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: osdmap e13: 8 total, 0 up, 8 in 2023-06-24T09:29:02.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:02.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:02.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:02.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:02.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:02.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:02.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:02.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:02 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"num_osds":8,"num_up_osds":0,"osd_up_since":0,"num_in_osds":8,"osd_in_since":1687598941,"num_remapped_pgs":0} 2023-06-24T09:29:04.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:04 smithi188 ceph-mon[112326]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:04.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:04 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/2470754873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:29:04.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:04 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1817514136' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:04.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:04 smithi045 ceph-mon[102509]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:04.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:04 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2470754873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-24T09:29:04.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:04 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1817514136' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:05.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:29:05.327 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:06.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:06 smithi188 ceph-mon[112326]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:06.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:06 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-06-24T09:29:06.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:06 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:06.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:06 smithi045 ceph-mon[102509]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:06.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:06 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-06-24T09:29:06.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:06 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:07.119 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:07.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:07 smithi188 ceph-mon[112326]: Deploying daemon osd.0 on smithi188 2023-06-24T09:29:07.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:07 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/77048204' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:07.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:07 smithi045 ceph-mon[102509]: Deploying daemon osd.0 on smithi188 2023-06-24T09:29:07.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:07 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/77048204' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:07.898 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"num_osds":8,"num_up_osds":0,"osd_up_since":0,"num_in_osds":8,"osd_in_since":1687598941,"num_remapped_pgs":0} 2023-06-24T09:29:08.156 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:08 smithi045 ceph-mon[102509]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:08.156 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:29:08.189 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:08 smithi188 ceph-mon[112326]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:08.189 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:29:08.899 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:29:09.155 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:09.391 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:09 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-06-24T09:29:09.391 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:09 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:09.593 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:09 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-06-24T09:29:09.593 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:09 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:10.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:10 smithi188 ceph-mon[112326]: Deploying daemon osd.1 on smithi045 2023-06-24T09:29:10.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:10 smithi188 ceph-mon[112326]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:10.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:10 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:10.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:10 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:10.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:10 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-06-24T09:29:10.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:10 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:10.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:10 smithi045 ceph-mon[102509]: Deploying daemon osd.1 on smithi045 2023-06-24T09:29:10.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:10 smithi045 ceph-mon[102509]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:10.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:10 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:10.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:10 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:10.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:10 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-06-24T09:29:10.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:10 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:11.029 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:11.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:11 smithi045 ceph-mon[102509]: Deploying daemon osd.2 on smithi188 2023-06-24T09:29:11.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:11 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/676162632' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:11.509 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:11 smithi188 ceph-mon[112326]: Deploying daemon osd.2 on smithi188 2023-06-24T09:29:11.509 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:11 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/676162632' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:11.782 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"num_osds":8,"num_up_osds":0,"osd_up_since":0,"num_in_osds":8,"osd_in_since":1687598941,"num_remapped_pgs":0} 2023-06-24T09:29:12.329 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:12 smithi045 ceph-mon[102509]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:12.454 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:12 smithi188 ceph-mon[112326]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:12.784 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:29:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:13.402 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:13 smithi045 ceph-mon[102509]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:13.533 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:13 smithi188 ceph-mon[112326]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:14.396 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:14 smithi188 ceph-mon[112326]: from='osd.0 [v2:172.21.15.188:6800/1318724509,v1:172.21.15.188:6801/1318724509]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-24T09:29:14.397 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:14 smithi188 ceph-mon[112326]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-24T09:29:14.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:14 smithi045 ceph-mon[102509]: from='osd.0 [v2:172.21.15.188:6800/1318724509,v1:172.21.15.188:6801/1318724509]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-24T09:29:14.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:14 smithi045 ceph-mon[102509]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-24T09:29:14.697 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":14,"num_osds":8,"num_up_osds":0,"osd_up_since":0,"num_in_osds":8,"osd_in_since":1687598941,"num_remapped_pgs":0} 2023-06-24T09:29:15.382 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-06-24T09:29:15.382 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: osdmap e14: 8 total, 0 up, 8 in 2023-06-24T09:29:15.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:15.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:15.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:15.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:15.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:15.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:15.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='osd.0 [v2:172.21.15.188:6800/1318724509,v1:172.21.15.188:6801/1318724509]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:15.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:15.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:15.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:15.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:15.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:15.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-06-24T09:29:15.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:15.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: Deploying daemon osd.4 on smithi188 2023-06-24T09:29:15.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3890671080' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:15.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:15.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:15.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-06-24T09:29:15.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:15.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: Deploying daemon osd.3 on smithi045 2023-06-24T09:29:15.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:15 smithi045 ceph-mon[102509]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:15.569 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-06-24T09:29:15.569 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: osdmap e14: 8 total, 0 up, 8 in 2023-06-24T09:29:15.569 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:15.570 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:15.570 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:15.570 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:15.570 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:15.570 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:15.570 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='osd.0 [v2:172.21.15.188:6800/1318724509,v1:172.21.15.188:6801/1318724509]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:15.571 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:15.571 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:15.571 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:15.571 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:15.571 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:15.571 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-06-24T09:29:15.571 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:15.572 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: Deploying daemon osd.4 on smithi188 2023-06-24T09:29:15.572 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3890671080' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:15.572 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:15.572 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:15.572 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-06-24T09:29:15.572 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:15.573 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: Deploying daemon osd.3 on smithi045 2023-06-24T09:29:15.573 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:15 smithi188 ceph-mon[112326]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:16.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:29:16.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:16 smithi045 ceph-mon[102509]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2023-06-24T09:29:16.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:16 smithi045 ceph-mon[102509]: osdmap e15: 8 total, 0 up, 8 in 2023-06-24T09:29:16.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:16 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:16.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:16 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:16.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:16 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:16.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:16 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:16.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:16 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:16.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:16 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:16.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:16 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:16.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:16 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:16.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:16 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:16.597 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:16 smithi188 ceph-mon[112326]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2023-06-24T09:29:16.597 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:16 smithi188 ceph-mon[112326]: osdmap e15: 8 total, 0 up, 8 in 2023-06-24T09:29:16.597 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:16 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:16.597 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:16 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:16.598 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:16 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:16.598 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:16 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:16.598 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:16 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:16.598 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:16 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:16.598 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:16 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:16.598 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:16 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:16.598 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:16 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:17.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: purged_snaps scrub starts 2023-06-24T09:29:17.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: purged_snaps scrub ok 2023-06-24T09:29:17.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:17.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: osd.0 [v2:172.21.15.188:6800/1318724509,v1:172.21.15.188:6801/1318724509] boot 2023-06-24T09:29:17.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: osdmap e16: 8 total, 1 up, 8 in 2023-06-24T09:29:17.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:17.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:17.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:17.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:17.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:17.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:17.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:17.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:17.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:17 smithi045 ceph-mon[102509]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:17.630 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: purged_snaps scrub starts 2023-06-24T09:29:17.630 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: purged_snaps scrub ok 2023-06-24T09:29:17.630 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:17.630 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: osd.0 [v2:172.21.15.188:6800/1318724509,v1:172.21.15.188:6801/1318724509] boot 2023-06-24T09:29:17.630 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: osdmap e16: 8 total, 1 up, 8 in 2023-06-24T09:29:17.630 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-24T09:29:17.631 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:17.631 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:17.631 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:17.631 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:17.631 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:17.631 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:17.631 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:17.631 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:17 smithi188 ceph-mon[112326]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-24T09:29:18.143 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:18.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: osdmap e17: 8 total, 1 up, 8 in 2023-06-24T09:29:18.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:18.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:18.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:18.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:18.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:18.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:18.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:18.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: from='osd.2 [v2:172.21.15.188:6808/3175434033,v1:172.21.15.188:6809/3175434033]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-24T09:29:18.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-24T09:29:18.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: from='osd.1 [v2:172.21.15.45:6802/668241030,v1:172.21.15.45:6803/668241030]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-24T09:29:18.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:18 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1846790750' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:18.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: osdmap e17: 8 total, 1 up, 8 in 2023-06-24T09:29:18.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:18.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:18.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:18.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:18.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:18.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:18.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:18.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: from='osd.2 [v2:172.21.15.188:6808/3175434033,v1:172.21.15.188:6809/3175434033]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-24T09:29:18.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-24T09:29:18.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: from='osd.1 [v2:172.21.15.45:6802/668241030,v1:172.21.15.45:6803/668241030]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-24T09:29:18.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:18 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1846790750' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:18.591 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"num_osds":8,"num_up_osds":1,"osd_up_since":1687598956,"num_in_osds":8,"osd_in_since":1687598941,"num_remapped_pgs":0} 2023-06-24T09:29:19.451 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-06-24T09:29:19.451 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='osd.1 [v2:172.21.15.45:6802/668241030,v1:172.21.15.45:6803/668241030]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-06-24T09:29:19.452 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: osdmap e18: 8 total, 1 up, 8 in 2023-06-24T09:29:19.452 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:19.452 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:19.452 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='osd.1 [v2:172.21.15.45:6802/668241030,v1:172.21.15.45:6803/668241030]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-06-24T09:29:19.452 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:19.452 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:19.453 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:19.453 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:19.453 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:19.453 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='osd.2 [v2:172.21.15.188:6808/3175434033,v1:172.21.15.188:6809/3175434033]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:19.453 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:19.453 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:19.453 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:19.453 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-06-24T09:29:19.454 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:19.454 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: Deploying daemon osd.6 on smithi188 2023-06-24T09:29:19.454 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:19 smithi188 ceph-mon[112326]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-24T09:29:19.551 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-06-24T09:29:19.552 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='osd.1 [v2:172.21.15.45:6802/668241030,v1:172.21.15.45:6803/668241030]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-06-24T09:29:19.552 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: osdmap e18: 8 total, 1 up, 8 in 2023-06-24T09:29:19.552 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:19.552 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:19.553 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='osd.1 [v2:172.21.15.45:6802/668241030,v1:172.21.15.45:6803/668241030]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-06-24T09:29:19.553 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:19.553 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:19.553 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:19.553 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:19.554 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:19.554 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='osd.2 [v2:172.21.15.188:6808/3175434033,v1:172.21.15.188:6809/3175434033]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:19.554 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:19.554 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:19.554 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:19.555 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-06-24T09:29:19.555 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:19.555 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: Deploying daemon osd.6 on smithi188 2023-06-24T09:29:19.555 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:19 smithi045 ceph-mon[102509]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-24T09:29:19.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:29:19.853 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:20.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='osd.1 [v2:172.21.15.45:6802/668241030,v1:172.21.15.45:6803/668241030]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-06-24T09:29:20.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2023-06-24T09:29:20.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: osdmap e19: 8 total, 1 up, 8 in 2023-06-24T09:29:20.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:20.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:20.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:20.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:20.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:20.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:20.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:20.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:20.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:20.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:20.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:20.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-06-24T09:29:20.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:20.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: Deploying daemon osd.5 on smithi045 2023-06-24T09:29:20.424 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:20.588 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='osd.1 [v2:172.21.15.45:6802/668241030,v1:172.21.15.45:6803/668241030]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-06-24T09:29:20.588 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2023-06-24T09:29:20.588 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: osdmap e19: 8 total, 1 up, 8 in 2023-06-24T09:29:20.588 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:20.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:20.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:20.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:20.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:20.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:20.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:20.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:20.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:20.590 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:20.590 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:20.590 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-06-24T09:29:20.590 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:20.590 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: Deploying daemon osd.5 on smithi045 2023-06-24T09:29:20.590 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:21.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: purged_snaps scrub starts 2023-06-24T09:29:21.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: purged_snaps scrub ok 2023-06-24T09:29:21.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: purged_snaps scrub starts 2023-06-24T09:29:21.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: purged_snaps scrub ok 2023-06-24T09:29:21.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: osd.1 [v2:172.21.15.45:6802/668241030,v1:172.21.15.45:6803/668241030] boot 2023-06-24T09:29:21.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: osd.2 [v2:172.21.15.188:6808/3175434033,v1:172.21.15.188:6809/3175434033] boot 2023-06-24T09:29:21.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: osdmap e20: 8 total, 3 up, 8 in 2023-06-24T09:29:21.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:21.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:21.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:21.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:21.635 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:21.635 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:21.635 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:21 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:21.773 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: purged_snaps scrub starts 2023-06-24T09:29:21.774 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: purged_snaps scrub ok 2023-06-24T09:29:21.774 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: purged_snaps scrub starts 2023-06-24T09:29:21.774 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: purged_snaps scrub ok 2023-06-24T09:29:21.774 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: osd.1 [v2:172.21.15.45:6802/668241030,v1:172.21.15.45:6803/668241030] boot 2023-06-24T09:29:21.775 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: osd.2 [v2:172.21.15.188:6808/3175434033,v1:172.21.15.188:6809/3175434033] boot 2023-06-24T09:29:21.775 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: osdmap e20: 8 total, 3 up, 8 in 2023-06-24T09:29:21.775 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-24T09:29:21.775 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-24T09:29:21.776 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:21.776 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:21.776 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:21.777 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:21.777 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:21 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:22.492 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-24T09:29:22.492 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-06-24T09:29:22.493 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-06-24T09:29:22.493 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: osdmap e21: 8 total, 3 up, 8 in 2023-06-24T09:29:22.493 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:22.493 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:22.493 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:22.493 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:22.493 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:22.494 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-06-24T09:29:22.494 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2165853871' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:22.494 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: from='osd.4 [v2:172.21.15.188:6816/2271246416,v1:172.21.15.188:6817/2271246416]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-24T09:29:22.494 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:22 smithi045 ceph-mon[102509]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-24T09:29:22.510 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"num_osds":8,"num_up_osds":3,"osd_up_since":1687598960,"num_in_osds":8,"osd_in_since":1687598941,"num_remapped_pgs":0} 2023-06-24T09:29:22.732 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-24T09:29:22.732 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-06-24T09:29:22.733 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-06-24T09:29:22.733 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: osdmap e21: 8 total, 3 up, 8 in 2023-06-24T09:29:22.733 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:22.733 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:22.733 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:22.733 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:22.733 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:22.734 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-06-24T09:29:22.734 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/2165853871' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:22.734 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: from='osd.4 [v2:172.21.15.188:6816/2271246416,v1:172.21.15.188:6817/2271246416]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-24T09:29:22.734 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:22 smithi188 ceph-mon[112326]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-24T09:29:23.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:29:23.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-06-24T09:29:23.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-06-24T09:29:23.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: osdmap e22: 8 total, 3 up, 8 in 2023-06-24T09:29:23.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:23.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:23.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:23.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:23.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:23.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='osd.4 [v2:172.21.15.188:6816/2271246416,v1:172.21.15.188:6817/2271246416]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:23.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:23.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='osd.3 [v2:172.21.15.45:6810/2310265302,v1:172.21.15.45:6811/2310265302]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-24T09:29:23.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: pgmap v55: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-06-24T09:29:23.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:29:23.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:23.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:23.635 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:23.778 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-06-24T09:29:23.778 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-06-24T09:29:23.779 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: osdmap e22: 8 total, 3 up, 8 in 2023-06-24T09:29:23.779 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:23.779 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:23.779 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:23.779 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:23.779 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:23.779 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='osd.4 [v2:172.21.15.188:6816/2271246416,v1:172.21.15.188:6817/2271246416]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:23.780 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:23.780 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='osd.3 [v2:172.21.15.45:6810/2310265302,v1:172.21.15.45:6811/2310265302]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-24T09:29:23.780 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: pgmap v55: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-06-24T09:29:23.780 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:29:23.780 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:23.780 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:23.780 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:23.781 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:24.661 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:24 smithi188 ceph-mon[112326]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2023-06-24T09:29:24.662 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:24 smithi188 ceph-mon[112326]: from='osd.3 [v2:172.21.15.45:6810/2310265302,v1:172.21.15.45:6811/2310265302]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-06-24T09:29:24.662 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:24 smithi188 ceph-mon[112326]: osdmap e23: 8 total, 3 up, 8 in 2023-06-24T09:29:24.662 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:24.662 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:24.662 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:24.662 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:24.662 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:24 smithi188 ceph-mon[112326]: from='osd.3 [v2:172.21.15.45:6810/2310265302,v1:172.21.15.45:6811/2310265302]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-06-24T09:29:24.663 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:24.663 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:24.663 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:24 smithi188 ceph-mon[112326]: mgrmap e19: smithi045.dcgntd(active, since 91s), standbys: smithi188.zipfzj 2023-06-24T09:29:24.893 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:24 smithi045 ceph-mon[102509]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2023-06-24T09:29:24.894 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:24 smithi045 ceph-mon[102509]: from='osd.3 [v2:172.21.15.45:6810/2310265302,v1:172.21.15.45:6811/2310265302]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-06-24T09:29:24.894 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:24 smithi045 ceph-mon[102509]: osdmap e23: 8 total, 3 up, 8 in 2023-06-24T09:29:24.894 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:24.894 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:24.894 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:24.895 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:24.895 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:24 smithi045 ceph-mon[102509]: from='osd.3 [v2:172.21.15.45:6810/2310265302,v1:172.21.15.45:6811/2310265302]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-06-24T09:29:24.895 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:24.895 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:24.895 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:24 smithi045 ceph-mon[102509]: mgrmap e19: smithi045.dcgntd(active, since 91s), standbys: smithi188.zipfzj 2023-06-24T09:29:25.604 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:25.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: purged_snaps scrub starts 2023-06-24T09:29:25.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: purged_snaps scrub ok 2023-06-24T09:29:25.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: from='osd.3 [v2:172.21.15.45:6810/2310265302,v1:172.21.15.45:6811/2310265302]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-06-24T09:29:25.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: osd.4 [v2:172.21.15.188:6816/2271246416,v1:172.21.15.188:6817/2271246416] boot 2023-06-24T09:29:25.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: osdmap e24: 8 total, 4 up, 8 in 2023-06-24T09:29:25.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:25.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:25.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:25.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:25.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:25.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:25.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: pgmap v58: 1 pgs: 1 unknown; 0 B data, 106 MiB used, 357 GiB / 358 GiB avail 2023-06-24T09:29:25.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: osd.3 [v2:172.21.15.45:6810/2310265302,v1:172.21.15.45:6811/2310265302] boot 2023-06-24T09:29:25.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:25 smithi188 ceph-mon[112326]: osdmap e25: 8 total, 5 up, 8 in 2023-06-24T09:29:25.788 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: purged_snaps scrub starts 2023-06-24T09:29:25.788 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: purged_snaps scrub ok 2023-06-24T09:29:25.789 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: from='osd.3 [v2:172.21.15.45:6810/2310265302,v1:172.21.15.45:6811/2310265302]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-06-24T09:29:25.789 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: osd.4 [v2:172.21.15.188:6816/2271246416,v1:172.21.15.188:6817/2271246416] boot 2023-06-24T09:29:25.789 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: osdmap e24: 8 total, 4 up, 8 in 2023-06-24T09:29:25.789 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:25.789 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-24T09:29:25.789 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:25.789 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:25.789 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:25.790 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:25.790 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: pgmap v58: 1 pgs: 1 unknown; 0 B data, 106 MiB used, 357 GiB / 358 GiB avail 2023-06-24T09:29:25.790 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: osd.3 [v2:172.21.15.45:6810/2310265302,v1:172.21.15.45:6811/2310265302] boot 2023-06-24T09:29:25.790 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:25 smithi045 ceph-mon[102509]: osdmap e25: 8 total, 5 up, 8 in 2023-06-24T09:29:26.056 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"num_osds":8,"num_up_osds":5,"osd_up_since":1687598965,"num_in_osds":8,"osd_in_since":1687598941,"num_remapped_pgs":1} 2023-06-24T09:29:26.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: purged_snaps scrub starts 2023-06-24T09:29:26.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: purged_snaps scrub ok 2023-06-24T09:29:26.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:26.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:26.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:26.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:26.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1339990559' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:26.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:26.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:26.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-06-24T09:29:26.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:26.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: Deploying daemon osd.7 on smithi045 2023-06-24T09:29:26.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:26.635 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:26.635 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: osdmap e26: 8 total, 5 up, 8 in 2023-06-24T09:29:26.635 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:26.635 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:26.635 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:26.635 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='osd.6 [v2:172.21.15.188:6824/55846086,v1:172.21.15.188:6825/55846086]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-24T09:29:26.635 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:26 smithi188 ceph-mon[112326]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-24T09:29:26.650 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: purged_snaps scrub starts 2023-06-24T09:29:26.651 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: purged_snaps scrub ok 2023-06-24T09:29:26.651 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-24T09:29:26.651 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:26.651 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:26.651 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:26.652 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1339990559' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:26.652 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:26.652 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:26.652 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-06-24T09:29:26.653 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:26.653 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: Deploying daemon osd.7 on smithi045 2023-06-24T09:29:26.653 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:26.654 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:26.654 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: osdmap e26: 8 total, 5 up, 8 in 2023-06-24T09:29:26.654 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:26.654 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:26.654 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:26.655 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='osd.6 [v2:172.21.15.188:6824/55846086,v1:172.21.15.188:6825/55846086]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-24T09:29:26.655 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:26 smithi045 ceph-mon[102509]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-24T09:29:27.059 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:29:27.328 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:27.716 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:27 smithi045 ceph-mon[102509]: pgmap v61: 1 pgs: 1 remapped; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-06-24T09:29:27.716 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:27 smithi045 ceph-mon[102509]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-06-24T09:29:27.716 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:27 smithi045 ceph-mon[102509]: osdmap e27: 8 total, 5 up, 8 in 2023-06-24T09:29:27.716 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:27 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:27.717 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:27 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:27.717 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:27 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:27.717 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:27 smithi045 ceph-mon[102509]: from='osd.6 [v2:172.21.15.188:6824/55846086,v1:172.21.15.188:6825/55846086]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:27.717 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:27 smithi045 ceph-mon[102509]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:27.834 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:27 smithi188 ceph-mon[112326]: pgmap v61: 1 pgs: 1 remapped; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-06-24T09:29:27.835 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:27 smithi188 ceph-mon[112326]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-06-24T09:29:27.835 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:27 smithi188 ceph-mon[112326]: osdmap e27: 8 total, 5 up, 8 in 2023-06-24T09:29:27.835 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:27 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:27.836 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:27 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:27.836 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:27 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:27.836 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:27 smithi188 ceph-mon[112326]: from='osd.6 [v2:172.21.15.188:6824/55846086,v1:172.21.15.188:6825/55846086]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:27.836 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:27 smithi188 ceph-mon[112326]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2023-06-24T09:29:28.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:28 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:28.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:28 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:28.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:28 smithi188 ceph-mon[112326]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2023-06-24T09:29:28.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:28 smithi188 ceph-mon[112326]: osdmap e28: 8 total, 5 up, 8 in 2023-06-24T09:29:28.885 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:28 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:28.885 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:28 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:28.885 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:28 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:28.886 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:28 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:28.886 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:28 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:28.887 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:28 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:28.887 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:28 smithi045 ceph-mon[102509]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2023-06-24T09:29:28.887 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:28 smithi045 ceph-mon[102509]: osdmap e28: 8 total, 5 up, 8 in 2023-06-24T09:29:28.888 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:28 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:28.888 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:28 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:28.888 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:28 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:28.888 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:28 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:29.077 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:29.805 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"num_osds":8,"num_up_osds":5,"osd_up_since":1687598965,"num_in_osds":8,"osd_in_since":1687598941,"num_remapped_pgs":0} 2023-06-24T09:29:29.813 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: purged_snaps scrub starts 2023-06-24T09:29:29.813 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: purged_snaps scrub ok 2023-06-24T09:29:29.813 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: from='osd.5 [v2:172.21.15.45:6818/1721914718,v1:172.21.15.45:6819/1721914718]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-24T09:29:29.813 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-24T09:29:29.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: pgmap v64: 1 pgs: 1 remapped; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-06-24T09:29:29.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:29.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1830605475' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:29.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: osd.6 [v2:172.21.15.188:6824/55846086,v1:172.21.15.188:6825/55846086] boot 2023-06-24T09:29:29.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-06-24T09:29:29.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: osdmap e29: 8 total, 6 up, 8 in 2023-06-24T09:29:29.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: from='osd.5 [v2:172.21.15.45:6818/1721914718,v1:172.21.15.45:6819/1721914718]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-06-24T09:29:29.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-06-24T09:29:29.815 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:29.815 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:29.815 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:29.815 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:29 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:29.908 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: purged_snaps scrub starts 2023-06-24T09:29:29.909 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: purged_snaps scrub ok 2023-06-24T09:29:29.909 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: from='osd.5 [v2:172.21.15.45:6818/1721914718,v1:172.21.15.45:6819/1721914718]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-24T09:29:29.909 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-24T09:29:29.909 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: pgmap v64: 1 pgs: 1 remapped; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-06-24T09:29:29.909 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:29.910 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1830605475' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:29.910 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: osd.6 [v2:172.21.15.188:6824/55846086,v1:172.21.15.188:6825/55846086] boot 2023-06-24T09:29:29.910 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-06-24T09:29:29.910 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: osdmap e29: 8 total, 6 up, 8 in 2023-06-24T09:29:29.910 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: from='osd.5 [v2:172.21.15.45:6818/1721914718,v1:172.21.15.45:6819/1721914718]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-06-24T09:29:29.910 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-06-24T09:29:29.910 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:29.910 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-24T09:29:29.911 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:29.911 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:29 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:30.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:29:30.838 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:30 smithi045 sudo[132238]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-24T09:29:30.838 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:30 smithi045 sudo[132238]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-24T09:29:30.838 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:30 smithi045 sudo[132238]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-24T09:29:30.838 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:30 smithi045 sudo[132238]: pam_unix(sudo:session): session closed for user root 2023-06-24T09:29:31.076 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:31.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:30 smithi045 sudo[132305]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-24T09:29:31.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:30 smithi045 sudo[132305]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-24T09:29:31.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:30 smithi045 sudo[132305]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-24T09:29:31.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 sudo[132305]: pam_unix(sudo:session): session closed for user root 2023-06-24T09:29:31.537 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 sudo[123944]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-24T09:29:31.538 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 sudo[123944]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-24T09:29:31.538 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 sudo[123944]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-24T09:29:31.538 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 sudo[123944]: pam_unix(sudo:session): session closed for user root 2023-06-24T09:29:31.538 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 sudo[123947]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-24T09:29:31.539 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 sudo[123947]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-24T09:29:31.539 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 sudo[123947]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-24T09:29:31.539 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 sudo[123947]: pam_unix(sudo:session): session closed for user root 2023-06-24T09:29:31.540 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:31.540 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-24T09:29:31.541 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-24T09:29:31.541 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi045"}]: dispatch 2023-06-24T09:29:31.541 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:29:31.541 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-06-24T09:29:31.542 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: osdmap e30: 8 total, 6 up, 8 in 2023-06-24T09:29:31.542 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:31.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:31.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-24T09:29:31.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-24T09:29:31.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi045"}]: dispatch 2023-06-24T09:29:31.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:29:31.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-06-24T09:29:31.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: osdmap e30: 8 total, 6 up, 8 in 2023-06-24T09:29:31.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:31.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:31.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:31.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:31.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:31.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:31.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi045"}]: dispatch 2023-06-24T09:29:31.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:31 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:29:31.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:31.924 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:31.925 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:31.925 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:31.925 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:31.925 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi045"}]: dispatch 2023-06-24T09:29:31.926 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:31 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "mon metadata", "id": "smithi188"}]: dispatch 2023-06-24T09:29:32.652 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:32 smithi045 ceph-mon[102509]: purged_snaps scrub starts 2023-06-24T09:29:32.653 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:32 smithi045 ceph-mon[102509]: purged_snaps scrub ok 2023-06-24T09:29:32.653 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:32 smithi045 ceph-mon[102509]: pgmap v66: 1 pgs: 1 remapped; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2023-06-24T09:29:32.653 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:32 smithi045 ceph-mon[102509]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-24T09:29:32.653 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:32 smithi045 ceph-mon[102509]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-24T09:29:32.653 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:32 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:32.653 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:32 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:32.654 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:32 smithi045 ceph-mon[102509]: osd.5 [v2:172.21.15.45:6818/1721914718,v1:172.21.15.45:6819/1721914718] boot 2023-06-24T09:29:32.654 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:32 smithi045 ceph-mon[102509]: osdmap e31: 8 total, 7 up, 8 in 2023-06-24T09:29:32.654 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:32 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:32.655 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:32 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:32.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:32.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:32 smithi188 ceph-mon[112326]: purged_snaps scrub starts 2023-06-24T09:29:32.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:32 smithi188 ceph-mon[112326]: purged_snaps scrub ok 2023-06-24T09:29:32.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:32 smithi188 ceph-mon[112326]: pgmap v66: 1 pgs: 1 remapped; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2023-06-24T09:29:32.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:32 smithi188 ceph-mon[112326]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-24T09:29:32.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:32 smithi188 ceph-mon[112326]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-24T09:29:32.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:32 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:32.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:32 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:32.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:32 smithi188 ceph-mon[112326]: osd.5 [v2:172.21.15.45:6818/1721914718,v1:172.21.15.45:6819/1721914718] boot 2023-06-24T09:29:32.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:32 smithi188 ceph-mon[112326]: osdmap e31: 8 total, 7 up, 8 in 2023-06-24T09:29:32.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:32 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-24T09:29:32.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:32 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:33.418 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":31,"num_osds":8,"num_up_osds":7,"osd_up_since":1687598972,"num_in_osds":8,"osd_in_since":1687598941,"num_remapped_pgs":0} 2023-06-24T09:29:33.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:33 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/684623896' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:33.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:33 smithi045 ceph-mon[102509]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 12 KiB/s rd, 695 KiB/s wr, 42 op/s; 128 KiB/s, 0 objects/s recovering 2023-06-24T09:29:33.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:33 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/684623896' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:33.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:33 smithi188 ceph-mon[112326]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 12 KiB/s rd, 695 KiB/s wr, 42 op/s; 128 KiB/s, 0 objects/s recovering 2023-06-24T09:29:34.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:29:34.673 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:34.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:34 smithi188 ceph-mon[112326]: osdmap e32: 8 total, 7 up, 8 in 2023-06-24T09:29:34.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:34 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:34.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:34 smithi188 ceph-mon[112326]: from='osd.7 [v2:172.21.15.45:6826/917484674,v1:172.21.15.45:6827/917484674]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-24T09:29:34.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:34 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:34.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:34 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:34.885 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:34 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:29:34.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:34 smithi045 ceph-mon[102509]: osdmap e32: 8 total, 7 up, 8 in 2023-06-24T09:29:34.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:34 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:34.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:34 smithi045 ceph-mon[102509]: from='osd.7 [v2:172.21.15.45:6826/917484674,v1:172.21.15.45:6827/917484674]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-24T09:29:34.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:34 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:34.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:34 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:34.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:34 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:29:35.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:35 smithi045 ceph-mon[102509]: from='osd.7 [v2:172.21.15.45:6826/917484674,v1:172.21.15.45:6827/917484674]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-06-24T09:29:35.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:35 smithi045 ceph-mon[102509]: osdmap e33: 8 total, 7 up, 8 in 2023-06-24T09:29:35.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:35 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:35.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:35 smithi045 ceph-mon[102509]: from='osd.7 [v2:172.21.15.45:6826/917484674,v1:172.21.15.45:6827/917484674]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-06-24T09:29:35.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:35 smithi045 ceph-mon[102509]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 13 KiB/s rd, 783 KiB/s wr, 47 op/s; 144 KiB/s, 0 objects/s recovering 2023-06-24T09:29:35.673 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:35 smithi045 ceph-mon[102509]: from='osd.7 [v2:172.21.15.45:6826/917484674,v1:172.21.15.45:6827/917484674]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-06-24T09:29:35.673 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:35 smithi045 ceph-mon[102509]: osdmap e34: 8 total, 7 up, 8 in 2023-06-24T09:29:35.673 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:35 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:35.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:35 smithi188 ceph-mon[112326]: from='osd.7 [v2:172.21.15.45:6826/917484674,v1:172.21.15.45:6827/917484674]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-06-24T09:29:35.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:35 smithi188 ceph-mon[112326]: osdmap e33: 8 total, 7 up, 8 in 2023-06-24T09:29:35.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:35 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:35.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:35 smithi188 ceph-mon[112326]: from='osd.7 [v2:172.21.15.45:6826/917484674,v1:172.21.15.45:6827/917484674]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2023-06-24T09:29:35.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:35 smithi188 ceph-mon[112326]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 13 KiB/s rd, 783 KiB/s wr, 47 op/s; 144 KiB/s, 0 objects/s recovering 2023-06-24T09:29:35.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:35 smithi188 ceph-mon[112326]: from='osd.7 [v2:172.21.15.45:6826/917484674,v1:172.21.15.45:6827/917484674]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2023-06-24T09:29:35.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:35 smithi188 ceph-mon[112326]: osdmap e34: 8 total, 7 up, 8 in 2023-06-24T09:29:35.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:35 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:36.239 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:36.734 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"num_osds":8,"num_up_osds":7,"osd_up_since":1687598972,"num_in_osds":8,"osd_in_since":1687598941,"num_remapped_pgs":0} 2023-06-24T09:29:36.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:36 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:36.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:36 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:36.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:36 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:36.814 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:36 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3494503287' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:36.815 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:36 smithi188 ceph-mon[112326]: osd.7 [v2:172.21.15.45:6826/917484674,v1:172.21.15.45:6827/917484674] boot 2023-06-24T09:29:36.815 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:36 smithi188 ceph-mon[112326]: osdmap e35: 8 total, 8 up, 8 in 2023-06-24T09:29:36.815 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:36 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:36.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:36 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:36.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:36 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:36.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:36 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:36.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:36 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3494503287' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:36.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:36 smithi045 ceph-mon[102509]: osd.7 [v2:172.21.15.45:6826/917484674,v1:172.21.15.45:6827/917484674] boot 2023-06-24T09:29:36.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:36 smithi045 ceph-mon[102509]: osdmap e35: 8 total, 8 up, 8 in 2023-06-24T09:29:36.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:36 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-24T09:29:37.572 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:37 smithi188 ceph-mon[112326]: purged_snaps scrub starts 2023-06-24T09:29:37.572 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:37 smithi188 ceph-mon[112326]: purged_snaps scrub ok 2023-06-24T09:29:37.572 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:37 smithi188 ceph-mon[112326]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:29:37.572 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:37 smithi188 ceph-mon[112326]: osdmap e36: 8 total, 8 up, 8 in 2023-06-24T09:29:37.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-24T09:29:37.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:37 smithi045 ceph-mon[102509]: purged_snaps scrub starts 2023-06-24T09:29:37.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:37 smithi045 ceph-mon[102509]: purged_snaps scrub ok 2023-06-24T09:29:37.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:37 smithi045 ceph-mon[102509]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:29:37.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:37 smithi045 ceph-mon[102509]: osdmap e36: 8 total, 8 up, 8 in 2023-06-24T09:29:37.987 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:38.872 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:29:38.873 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:38.873 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:38.873 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:38 smithi188 ceph-mon[112326]: osdmap e37: 8 total, 8 up, 8 in 2023-06-24T09:29:38.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:29:38.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:38.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:38.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:38 smithi045 ceph-mon[102509]: osdmap e37: 8 total, 8 up, 8 in 2023-06-24T09:29:39.372 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":37,"num_osds":8,"num_up_osds":8,"osd_up_since":1687598976,"num_in_osds":8,"osd_in_since":1687598941,"num_remapped_pgs":1} 2023-06-24T09:29:40.051 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd dump --format=json 2023-06-24T09:29:40.242 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: Detected new or changed devices on smithi188 2023-06-24T09:29:40.243 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:40.243 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:40.243 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:40.243 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:40.243 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:40.243 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:40.244 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: Adjusting osd_memory_target on smithi188 to 3736M 2023-06-24T09:29:40.244 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:40.244 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: pgmap v78: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2023-06-24T09:29:40.244 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3138807298' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:40.244 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:39 smithi045 ceph-mon[102509]: osdmap e38: 8 total, 8 up, 8 in 2023-06-24T09:29:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:40.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: Detected new or changed devices on smithi188 2023-06-24T09:29:40.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:40.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:40.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:40.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:40.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:40.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:40.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: Adjusting osd_memory_target on smithi188 to 3736M 2023-06-24T09:29:40.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:40.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: pgmap v78: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2023-06-24T09:29:40.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3138807298' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-24T09:29:40.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:39 smithi188 ceph-mon[112326]: osdmap e38: 8 total, 8 up, 8 in 2023-06-24T09:29:41.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:41 smithi188 ceph-mon[112326]: osdmap e39: 8 total, 8 up, 8 in 2023-06-24T09:29:41.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:41 smithi188 ceph-mon[112326]: pgmap v81: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2023-06-24T09:29:41.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:41 smithi045 ceph-mon[102509]: osdmap e39: 8 total, 8 up, 8 in 2023-06-24T09:29:41.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:41 smithi045 ceph-mon[102509]: pgmap v81: 1 pgs: 1 remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2023-06-24T09:29:42.061 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:42.062 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","created":"2023-06-24T09:25:36.936739+0000","modified":"2023-06-24T09:29:40.559880+0000","last_up_change":"2023-06-24T09:29:36.467459+0000","last_in_change":"2023-06-24T09:29:01.547318+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":15,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-24T09:29:21.463475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"602c763e-3002-4a3a-8243-abf07f5096b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6801","nonce":1318724509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6803","nonce":1318724509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6807","nonce":1318724509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6805","nonce":1318724509}]},"public_addr":"172.21.15.188:6801/1318724509","cluster_addr":"172.21.15.188:6803/1318724509","heartbeat_back_addr":"172.21.15.188:6807/1318724509","heartbeat_front_addr":"172.21.15.188:6805/1318724509","state":["exists","up"]},{"osd":1,"uuid":"1c41d63a-21f0-429d-8ca5-0d07abdb8b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6803","nonce":668241030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6805","nonce":668241030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6809","nonce":668241030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6807","nonce":668241030}]},"public_addr":"172.21.15.45:6803/668241030","cluster_addr":"172.21.15.45:6805/668241030","heartbeat_back_addr":"172.21.15.45:6809/668241030","heartbeat_front_addr":"172.21.15.45:6807/668241030","state":["exists","up"]},{"osd":2,"uuid":"313dc3fd-6227-4d95-a488-2f9d1368aa6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6809","nonce":3175434033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6811","nonce":3175434033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6815","nonce":3175434033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6813","nonce":3175434033}]},"public_addr":"172.21.15.188:6809/3175434033","cluster_addr":"172.21.15.188:6811/3175434033","heartbeat_back_addr":"172.21.15.188:6815/3175434033","heartbeat_front_addr":"172.21.15.188:6813/3175434033","state":["exists","up"]},{"osd":3,"uuid":"3c1a9355-00bc-4add-bac9-249ed07d0c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6811","nonce":2310265302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6813","nonce":2310265302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6817","nonce":2310265302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6815","nonce":2310265302}]},"public_addr":"172.21.15.45:6811/2310265302","cluster_addr":"172.21.15.45:6813/2310265302","heartbeat_back_addr":"172.21.15.45:6817/2310265302","heartbeat_front_addr":"172.21.15.45:6815/2310265302","state":["exists","up"]},{"osd":4,"uuid":"b392039a-ca73-47b7-9164-a9918a4c10f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6817","nonce":2271246416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6819","nonce":2271246416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6823","nonce":2271246416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6821","nonce":2271246416}]},"public_addr":"172.21.15.188:6817/2271246416","cluster_addr":"172.21.15.188:6819/2271246416","heartbeat_back_addr":"172.21.15.188:6823/2271246416","heartbeat_front_addr":"172.21.15.188:6821/2271246416","state":["exists","up"]},{"osd":5,"uuid":"cb292935-34ed-4c37-ad12-969cfee51cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6819","nonce":1721914718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6821","nonce":1721914718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6825","nonce":1721914718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6823","nonce":1721914718}]},"public_addr":"172.21.15.45:6819/1721914718","cluster_addr":"172.21.15.45:6821/1721914718","heartbeat_back_addr":"172.21.15.45:6825/1721914718","heartbeat_front_addr":"172.21.15.45:6823/1721914718","state":["exists","up"]},{"osd":6,"uuid":"e76afd0b-efa6-45bb-b611-18a5f7db84ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6825","nonce":55846086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6827","nonce":55846086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6831","nonce":55846086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6829","nonce":55846086}]},"public_addr":"172.21.15.188:6825/55846086","cluster_addr":"172.21.15.188:6827/55846086","heartbeat_back_addr":"172.21.15.188:6831/55846086","heartbeat_front_addr":"172.21.15.188:6829/55846086","state":["exists","up"]},{"osd":7,"uuid":"4290544c-7c88-4e40-b0b4-8abda55c5c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6827","nonce":917484674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6829","nonce":917484674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6833","nonce":917484674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6831","nonce":917484674}]},"public_addr":"172.21.15.45:6827/917484674","cluster_addr":"172.21.15.45:6829/917484674","heartbeat_back_addr":"172.21.15.45:6833/917484674","heartbeat_front_addr":"172.21.15.45:6831/917484674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:14.326373+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:19.007688+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:18.643422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:24.003526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:23.178615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:29.903901+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:27.423043+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:35.333269+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/2766041847":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/3411741152":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/2150819286":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/3042758172":"2023-06-25T09:27:52.956982+0000","172.21.15.45:6800/2766041847":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/893358506":"2023-06-25T09:26:30.889987+0000","172.21.15.45:6801/2227323743":"2023-06-25T09:26:30.889987+0000","172.21.15.45:0/306577999":"2023-06-25T09:26:30.889987+0000","172.21.15.45:6800/2180661052":"2023-06-25T09:26:02.885924+0000","172.21.15.45:6800/2227323743":"2023-06-25T09:26:30.889987+0000","172.21.15.45:6801/2180661052":"2023-06-25T09:26:02.885924+0000","172.21.15.45:0/1932085742":"2023-06-25T09:26:02.885924+0000","172.21.15.45:0/1440251954":"2023-06-25T09:26:02.885924+0000","172.21.15.45:0/2704290309":"2023-06-25T09:26:30.889987+0000","172.21.15.45:0/3258514032":"2023-06-25T09:26:02.885924+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-24T09:29:42.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:42 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/539682194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-24T09:29:42.704 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-24T09:29:21.463475+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-24T09:29:42.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-06-24T09:29:42.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:42 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/539682194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-24T09:29:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:44.112 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: Detected new or changed devices on smithi045 2023-06-24T09:29:44.113 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:44.113 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:44.113 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:44.113 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:44.113 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:44.114 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:44.114 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: Adjusting osd_memory_target on smithi045 to 2969M 2023-06-24T09:29:44.114 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:44.114 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:44.114 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:29:44.114 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:44.114 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:29:44.115 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-24T09:29:44.115 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:44.115 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-24T09:29:44.115 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:44.115 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:43 smithi188 ceph-mon[112326]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-06-24T09:29:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: Detected new or changed devices on smithi045 2023-06-24T09:29:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:29:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: Adjusting osd_memory_target on smithi045 to 2969M 2023-06-24T09:29:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:29:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:29:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-24T09:29:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-24T09:29:44.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:29:44.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:43 smithi045 ceph-mon[102509]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 105 KiB/s, 0 objects/s recovering 2023-06-24T09:29:44.687 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2023-06-24T09:29:45.037 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:44 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3096336523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-24T09:29:45.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:44 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3096336523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-24T09:29:45.258 INFO:tasks.cephadm:Setting up client nodes... 2023-06-24T09:29:45.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-06-24T09:29:45.516 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:46.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:45 smithi188 ceph-mon[112326]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-06-24T09:29:46.161 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:45 smithi045 ceph-mon[102509]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-06-24T09:29:47.399 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2023-06-24T09:29:47.400 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCLt5Zk/MyRFxAAul1uCPI+P6dislcrCkaAoQ== 2023-06-24T09:29:47.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:47 smithi045 ceph-mon[102509]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-24T09:29:47.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:47 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2728723952' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-06-24T09:29:47.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:47 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2728723952' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-06-24T09:29:47.848 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:47 smithi188 ceph-mon[112326]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-24T09:29:47.849 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:47 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/2728723952' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-06-24T09:29:47.849 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:47 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/2728723952' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-06-24T09:29:47.908 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:29:47.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-06-24T09:29:47.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-24T09:29:47.951 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-06-24T09:29:48.473 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi188/config 2023-06-24T09:29:50.078 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2023-06-24T09:29:50.078 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQCOt5Zki4djBBAAwJPweigAR/+xf+avyjTmkQ== 2023-06-24T09:29:50.107 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:50 smithi188 ceph-mon[112326]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-06-24T09:29:50.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:50 smithi045 ceph-mon[102509]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-06-24T09:29:50.586 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-06-24T09:29:50.586 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2023-06-24T09:29:50.587 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-06-24T09:29:50.626 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-06-24T09:29:50.627 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-06-24T09:29:50.628 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph mgr dump --format=json 2023-06-24T09:29:50.874 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:51.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:51 smithi188 ceph-mon[112326]: from='client.? 172.21.15.188:0/261666653' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-06-24T09:29:51.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:51 smithi188 ceph-mon[112326]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-06-24T09:29:51.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:51 smithi188 ceph-mon[112326]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-06-24T09:29:51.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:51 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:51.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:51 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:51.325 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:51 smithi045 ceph-mon[102509]: from='client.? 172.21.15.188:0/261666653' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-06-24T09:29:51.325 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:51 smithi045 ceph-mon[102509]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-06-24T09:29:51.326 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:51 smithi045 ceph-mon[102509]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-06-24T09:29:51.326 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:51 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:51.326 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:51 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:52.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:52 smithi188 ceph-mon[112326]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-06-24T09:29:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:52.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:52 smithi045 ceph-mon[102509]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-06-24T09:29:52.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:52.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:29:52.826 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:53.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:53 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1714008688' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-06-24T09:29:53.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:29:53.240 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":19,"active_gid":14229,"active_name":"smithi045.dcgntd","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1878561990},{"type":"v1","addr":"172.21.15.45:6801","nonce":1878561990}]},"active_addr":"172.21.15.45:6801/1878561990","active_change":"2023-06-24T09:27:52.957137+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14252,"name":"smithi188.zipfzj","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/","prometheus":"http://172.21.15.45:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3484226791}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1571635027}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":4035733855}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":168387098}]}]} 2023-06-24T09:29:53.246 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-06-24T09:29:53.246 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-06-24T09:29:53.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd dump --format=json 2023-06-24T09:29:53.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:53 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1714008688' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-06-24T09:29:53.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:29:53.484 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:54 smithi188 ceph-mon[112326]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-06-24T09:29:54.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:54 smithi045 ceph-mon[102509]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-06-24T09:29:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:54.959 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","created":"2023-06-24T09:25:36.936739+0000","modified":"2023-06-24T09:29:40.559880+0000","last_up_change":"2023-06-24T09:29:36.467459+0000","last_in_change":"2023-06-24T09:29:01.547318+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":15,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-24T09:29:21.463475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"602c763e-3002-4a3a-8243-abf07f5096b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6801","nonce":1318724509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6803","nonce":1318724509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6807","nonce":1318724509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6805","nonce":1318724509}]},"public_addr":"172.21.15.188:6801/1318724509","cluster_addr":"172.21.15.188:6803/1318724509","heartbeat_back_addr":"172.21.15.188:6807/1318724509","heartbeat_front_addr":"172.21.15.188:6805/1318724509","state":["exists","up"]},{"osd":1,"uuid":"1c41d63a-21f0-429d-8ca5-0d07abdb8b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6803","nonce":668241030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6805","nonce":668241030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6809","nonce":668241030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6807","nonce":668241030}]},"public_addr":"172.21.15.45:6803/668241030","cluster_addr":"172.21.15.45:6805/668241030","heartbeat_back_addr":"172.21.15.45:6809/668241030","heartbeat_front_addr":"172.21.15.45:6807/668241030","state":["exists","up"]},{"osd":2,"uuid":"313dc3fd-6227-4d95-a488-2f9d1368aa6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6809","nonce":3175434033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6811","nonce":3175434033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6815","nonce":3175434033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6813","nonce":3175434033}]},"public_addr":"172.21.15.188:6809/3175434033","cluster_addr":"172.21.15.188:6811/3175434033","heartbeat_back_addr":"172.21.15.188:6815/3175434033","heartbeat_front_addr":"172.21.15.188:6813/3175434033","state":["exists","up"]},{"osd":3,"uuid":"3c1a9355-00bc-4add-bac9-249ed07d0c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6811","nonce":2310265302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6813","nonce":2310265302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6817","nonce":2310265302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6815","nonce":2310265302}]},"public_addr":"172.21.15.45:6811/2310265302","cluster_addr":"172.21.15.45:6813/2310265302","heartbeat_back_addr":"172.21.15.45:6817/2310265302","heartbeat_front_addr":"172.21.15.45:6815/2310265302","state":["exists","up"]},{"osd":4,"uuid":"b392039a-ca73-47b7-9164-a9918a4c10f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6817","nonce":2271246416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6819","nonce":2271246416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6823","nonce":2271246416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6821","nonce":2271246416}]},"public_addr":"172.21.15.188:6817/2271246416","cluster_addr":"172.21.15.188:6819/2271246416","heartbeat_back_addr":"172.21.15.188:6823/2271246416","heartbeat_front_addr":"172.21.15.188:6821/2271246416","state":["exists","up"]},{"osd":5,"uuid":"cb292935-34ed-4c37-ad12-969cfee51cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6819","nonce":1721914718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6821","nonce":1721914718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6825","nonce":1721914718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6823","nonce":1721914718}]},"public_addr":"172.21.15.45:6819/1721914718","cluster_addr":"172.21.15.45:6821/1721914718","heartbeat_back_addr":"172.21.15.45:6825/1721914718","heartbeat_front_addr":"172.21.15.45:6823/1721914718","state":["exists","up"]},{"osd":6,"uuid":"e76afd0b-efa6-45bb-b611-18a5f7db84ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6825","nonce":55846086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6827","nonce":55846086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6831","nonce":55846086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6829","nonce":55846086}]},"public_addr":"172.21.15.188:6825/55846086","cluster_addr":"172.21.15.188:6827/55846086","heartbeat_back_addr":"172.21.15.188:6831/55846086","heartbeat_front_addr":"172.21.15.188:6829/55846086","state":["exists","up"]},{"osd":7,"uuid":"4290544c-7c88-4e40-b0b4-8abda55c5c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6827","nonce":917484674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6829","nonce":917484674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6833","nonce":917484674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6831","nonce":917484674}]},"public_addr":"172.21.15.45:6827/917484674","cluster_addr":"172.21.15.45:6829/917484674","heartbeat_back_addr":"172.21.15.45:6833/917484674","heartbeat_front_addr":"172.21.15.45:6831/917484674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:14.326373+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:19.007688+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:18.643422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:24.003526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:23.178615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:29.903901+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:27.423043+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:35.333269+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/2766041847":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/3411741152":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/2150819286":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/3042758172":"2023-06-25T09:27:52.956982+0000","172.21.15.45:6800/2766041847":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/893358506":"2023-06-25T09:26:30.889987+0000","172.21.15.45:6801/2227323743":"2023-06-25T09:26:30.889987+0000","172.21.15.45:0/306577999":"2023-06-25T09:26:30.889987+0000","172.21.15.45:6800/2180661052":"2023-06-25T09:26:02.885924+0000","172.21.15.45:6800/2227323743":"2023-06-25T09:26:30.889987+0000","172.21.15.45:6801/2180661052":"2023-06-25T09:26:02.885924+0000","172.21.15.45:0/1932085742":"2023-06-25T09:26:02.885924+0000","172.21.15.45:0/1440251954":"2023-06-25T09:26:02.885924+0000","172.21.15.45:0/2704290309":"2023-06-25T09:26:30.889987+0000","172.21.15.45:0/3258514032":"2023-06-25T09:26:02.885924+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-24T09:29:55.243 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:55 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1795890424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-24T09:29:55.340 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-06-24T09:29:55.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd dump --format=json 2023-06-24T09:29:55.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:55 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1795890424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-24T09:29:55.581 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:56.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:56 smithi045 ceph-mon[102509]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:29:56.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:56 smithi188 ceph-mon[112326]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:29:57.031 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:29:57.031 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"fsid":"fceaf654-1270-11ee-9b2b-001a4aab830c","created":"2023-06-24T09:25:36.936739+0000","modified":"2023-06-24T09:29:40.559880+0000","last_up_change":"2023-06-24T09:29:36.467459+0000","last_in_change":"2023-06-24T09:29:01.547318+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":15,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-24T09:29:21.463475+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"602c763e-3002-4a3a-8243-abf07f5096b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6801","nonce":1318724509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6803","nonce":1318724509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6807","nonce":1318724509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":1318724509},{"type":"v1","addr":"172.21.15.188:6805","nonce":1318724509}]},"public_addr":"172.21.15.188:6801/1318724509","cluster_addr":"172.21.15.188:6803/1318724509","heartbeat_back_addr":"172.21.15.188:6807/1318724509","heartbeat_front_addr":"172.21.15.188:6805/1318724509","state":["exists","up"]},{"osd":1,"uuid":"1c41d63a-21f0-429d-8ca5-0d07abdb8b0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6803","nonce":668241030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6805","nonce":668241030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6809","nonce":668241030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":668241030},{"type":"v1","addr":"172.21.15.45:6807","nonce":668241030}]},"public_addr":"172.21.15.45:6803/668241030","cluster_addr":"172.21.15.45:6805/668241030","heartbeat_back_addr":"172.21.15.45:6809/668241030","heartbeat_front_addr":"172.21.15.45:6807/668241030","state":["exists","up"]},{"osd":2,"uuid":"313dc3fd-6227-4d95-a488-2f9d1368aa6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6809","nonce":3175434033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6811","nonce":3175434033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6815","nonce":3175434033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3175434033},{"type":"v1","addr":"172.21.15.188:6813","nonce":3175434033}]},"public_addr":"172.21.15.188:6809/3175434033","cluster_addr":"172.21.15.188:6811/3175434033","heartbeat_back_addr":"172.21.15.188:6815/3175434033","heartbeat_front_addr":"172.21.15.188:6813/3175434033","state":["exists","up"]},{"osd":3,"uuid":"3c1a9355-00bc-4add-bac9-249ed07d0c94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6811","nonce":2310265302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6813","nonce":2310265302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6817","nonce":2310265302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2310265302},{"type":"v1","addr":"172.21.15.45:6815","nonce":2310265302}]},"public_addr":"172.21.15.45:6811/2310265302","cluster_addr":"172.21.15.45:6813/2310265302","heartbeat_back_addr":"172.21.15.45:6817/2310265302","heartbeat_front_addr":"172.21.15.45:6815/2310265302","state":["exists","up"]},{"osd":4,"uuid":"b392039a-ca73-47b7-9164-a9918a4c10f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6817","nonce":2271246416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6819","nonce":2271246416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6823","nonce":2271246416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2271246416},{"type":"v1","addr":"172.21.15.188:6821","nonce":2271246416}]},"public_addr":"172.21.15.188:6817/2271246416","cluster_addr":"172.21.15.188:6819/2271246416","heartbeat_back_addr":"172.21.15.188:6823/2271246416","heartbeat_front_addr":"172.21.15.188:6821/2271246416","state":["exists","up"]},{"osd":5,"uuid":"cb292935-34ed-4c37-ad12-969cfee51cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6819","nonce":1721914718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6821","nonce":1721914718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6825","nonce":1721914718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1721914718},{"type":"v1","addr":"172.21.15.45:6823","nonce":1721914718}]},"public_addr":"172.21.15.45:6819/1721914718","cluster_addr":"172.21.15.45:6821/1721914718","heartbeat_back_addr":"172.21.15.45:6825/1721914718","heartbeat_front_addr":"172.21.15.45:6823/1721914718","state":["exists","up"]},{"osd":6,"uuid":"e76afd0b-efa6-45bb-b611-18a5f7db84ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6825","nonce":55846086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6827","nonce":55846086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6831","nonce":55846086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":55846086},{"type":"v1","addr":"172.21.15.188:6829","nonce":55846086}]},"public_addr":"172.21.15.188:6825/55846086","cluster_addr":"172.21.15.188:6827/55846086","heartbeat_back_addr":"172.21.15.188:6831/55846086","heartbeat_front_addr":"172.21.15.188:6829/55846086","state":["exists","up"]},{"osd":7,"uuid":"4290544c-7c88-4e40-b0b4-8abda55c5c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6827","nonce":917484674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6829","nonce":917484674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6833","nonce":917484674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":917484674},{"type":"v1","addr":"172.21.15.45:6831","nonce":917484674}]},"public_addr":"172.21.15.45:6827/917484674","cluster_addr":"172.21.15.45:6829/917484674","heartbeat_back_addr":"172.21.15.45:6833/917484674","heartbeat_front_addr":"172.21.15.45:6831/917484674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:14.326373+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:19.007688+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:18.643422+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:24.003526+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:23.178615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:29.903901+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:27.423043+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-24T09:29:35.333269+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/2766041847":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/3411741152":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/2150819286":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/3042758172":"2023-06-25T09:27:52.956982+0000","172.21.15.45:6800/2766041847":"2023-06-25T09:27:52.956982+0000","172.21.15.45:0/893358506":"2023-06-25T09:26:30.889987+0000","172.21.15.45:6801/2227323743":"2023-06-25T09:26:30.889987+0000","172.21.15.45:0/306577999":"2023-06-25T09:26:30.889987+0000","172.21.15.45:6800/2180661052":"2023-06-25T09:26:02.885924+0000","172.21.15.45:6800/2227323743":"2023-06-25T09:26:30.889987+0000","172.21.15.45:6801/2180661052":"2023-06-25T09:26:02.885924+0000","172.21.15.45:0/1932085742":"2023-06-25T09:26:02.885924+0000","172.21.15.45:0/1440251954":"2023-06-25T09:26:02.885924+0000","172.21.15.45:0/2704290309":"2023-06-25T09:26:30.889987+0000","172.21.15.45:0/3258514032":"2023-06-25T09:26:02.885924+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-24T09:29:57.447 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-06-24T09:29:57.448 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-06-24T09:29:57.448 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-06-24T09:29:57.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-06-24T09:29:57.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-06-24T09:29:57.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-06-24T09:29:57.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-06-24T09:29:57.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-06-24T09:29:57.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:57 smithi188 ceph-mon[112326]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:29:57.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:29:57 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/940824946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-24T09:29:57.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:57 smithi045 ceph-mon[102509]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:29:57.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:29:57 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/940824946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-24T09:29:58.234 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:58.342 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:59.393 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:59.565 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:29:59.632 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:00.156 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:00.184 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:00 smithi045 ceph-mon[102509]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:00.185 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:00 smithi045 ceph-mon[102509]: overall HEALTH_OK 2023-06-24T09:30:00.347 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:00.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:00 smithi188 ceph-mon[112326]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:00.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:00 smithi188 ceph-mon[112326]: overall HEALTH_OK 2023-06-24T09:30:02.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:02 smithi188 ceph-mon[112326]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:02.394 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:02 smithi045 ceph-mon[102509]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:03.321 INFO:teuthology.orchestra.run.smithi045.stdout:107374182410 2023-06-24T09:30:03.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-06-24T09:30:04.071 INFO:teuthology.orchestra.run.smithi045.stdout:68719476749 2023-06-24T09:30:04.071 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-06-24T09:30:04.302 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:04 smithi045 ceph-mon[102509]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:04.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:04 smithi188 ceph-mon[112326]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:06.222 INFO:teuthology.orchestra.run.smithi045.stdout:124554051594 2023-06-24T09:30:06.222 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-06-24T09:30:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:06.348 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:06 smithi045 ceph-mon[102509]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:06.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:06 smithi188 ceph-mon[112326]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:06.536 INFO:teuthology.orchestra.run.smithi045.stdout:85899345931 2023-06-24T09:30:06.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-06-24T09:30:07.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:07 smithi045 ceph-mon[102509]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:07.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:07 smithi188 ceph-mon[112326]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:08.027 INFO:teuthology.orchestra.run.smithi045.stdout:150323855369 2023-06-24T09:30:08.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-06-24T09:30:08.076 INFO:teuthology.orchestra.run.smithi045.stdout:103079215115 2023-06-24T09:30:08.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-06-24T09:30:08.110 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:08.159 INFO:teuthology.orchestra.run.smithi045.stdout:85899345933 2023-06-24T09:30:08.160 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-06-24T09:30:08.679 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:30:08.808 INFO:teuthology.orchestra.run.smithi045.stdout:133143986185 2023-06-24T09:30:08.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-06-24T09:30:08.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:30:08.916 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:09.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:09 smithi045 ceph-mon[102509]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:09.760 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:09.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:09 smithi188 ceph-mon[112326]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:11.083 INFO:teuthology.orchestra.run.smithi045.stdout:107374182412 2023-06-24T09:30:11.404 INFO:teuthology.orchestra.run.smithi045.stdout:68719476750 2023-06-24T09:30:11.867 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:12.164 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:12 smithi045 ceph-mon[102509]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:12.165 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:12 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/370804624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-06-24T09:30:12.165 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:12 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/1041035227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-06-24T09:30:12.333 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476749 got 68719476750 for osd.0 2023-06-24T09:30:12.334 DEBUG:teuthology.parallel:result is None 2023-06-24T09:30:12.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:12 smithi188 ceph-mon[112326]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:12.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:12 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/370804624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-06-24T09:30:12.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:12 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/1041035227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-06-24T09:30:12.393 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182410 got 107374182412 for osd.3 2023-06-24T09:30:12.393 DEBUG:teuthology.parallel:result is None 2023-06-24T09:30:12.416 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:13.705 INFO:teuthology.orchestra.run.smithi045.stdout:85899345934 2023-06-24T09:30:14.120 INFO:teuthology.orchestra.run.smithi045.stdout:124554051595 2023-06-24T09:30:14.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:14 smithi045 ceph-mon[102509]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:14.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:14 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2426859614' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-06-24T09:30:14.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:14 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/524689496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-06-24T09:30:14.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:14 smithi188 ceph-mon[112326]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:14.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:14 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/2426859614' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-06-24T09:30:14.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:14 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/524689496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-06-24T09:30:14.599 INFO:tasks.cephadm.ceph_manager.ceph:need seq 85899345933 got 85899345934 for osd.1 2023-06-24T09:30:14.599 DEBUG:teuthology.parallel:result is None 2023-06-24T09:30:15.176 INFO:tasks.cephadm.ceph_manager.ceph:need seq 124554051594 got 124554051595 for osd.6 2023-06-24T09:30:15.176 DEBUG:teuthology.parallel:result is None 2023-06-24T09:30:15.675 INFO:teuthology.orchestra.run.smithi045.stdout:85899345933 2023-06-24T09:30:15.968 INFO:teuthology.orchestra.run.smithi045.stdout:150323855370 2023-06-24T09:30:16.308 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:16 smithi045 ceph-mon[102509]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:16.308 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:16 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2557242163' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-06-24T09:30:16.308 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:16 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/109204210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-06-24T09:30:16.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:16 smithi188 ceph-mon[112326]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:16.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:16 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/2557242163' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-06-24T09:30:16.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:16 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/109204210' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-06-24T09:30:16.604 INFO:tasks.cephadm.ceph_manager.ceph:need seq 85899345931 got 85899345933 for osd.2 2023-06-24T09:30:16.604 DEBUG:teuthology.parallel:result is None 2023-06-24T09:30:16.931 INFO:teuthology.orchestra.run.smithi045.stdout:103079215117 2023-06-24T09:30:17.023 INFO:teuthology.orchestra.run.smithi045.stdout:133143986187 2023-06-24T09:30:17.052 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855369 got 150323855370 for osd.7 2023-06-24T09:30:17.053 DEBUG:teuthology.parallel:result is None 2023-06-24T09:30:17.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:17 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2593727492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-06-24T09:30:17.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:17 smithi045 ceph-mon[102509]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:17.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:17 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/3223533306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-06-24T09:30:17.722 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986185 got 133143986187 for osd.5 2023-06-24T09:30:17.722 DEBUG:teuthology.parallel:result is None 2023-06-24T09:30:17.738 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215115 got 103079215117 for osd.4 2023-06-24T09:30:17.738 DEBUG:teuthology.parallel:result is None 2023-06-24T09:30:17.739 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-06-24T09:30:17.739 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph pg dump --format=json 2023-06-24T09:30:17.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:17 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/2593727492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-06-24T09:30:17.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:17 smithi188 ceph-mon[112326]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:17.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:17 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/3223533306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-06-24T09:30:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:19.345 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:30:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-06-24T09:30:19.744 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2023-06-24T09:30:18.983158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":154,"num_write":291,"num_write_kb":5548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":749928448,"kb_used":220412,"kb_used_data":4796,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":749708036,"statfs":{"total":767926730752,"available":767701028864,"internally_reserved":0,"allocated":4911104,"data_stored":3034970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001856"},"pg_stats":[{"pgid":"1.0","version":"30'237","reported_seq":339,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-24T09:29:40.566710+0000","last_change":"2023-06-24T09:29:40.566442+0000","last_active":"2023-06-24T09:29:40.566710+0000","last_peered":"2023-06-24T09:29:40.566710+0000","last_clean":"2023-06-24T09:29:40.566710+0000","last_became_active":"2023-06-24T09:29:40.566054+0000","last_became_peered":"2023-06-24T09:29:40.566054+0000","last_unstale":"2023-06-24T09:29:40.566710+0000","last_undegraded":"2023-06-24T09:29:40.566710+0000","last_fullsized":"2023-06-24T09:29:40.566710+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-24T09:29:21.465010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-24T09:29:21.465010+0000","last_clean_scrub_stamp":"2023-06-24T09:29:21.465010+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T20:46:59.569784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":154,"num_write":291,"num_write_kb":5548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":154,"num_write":291,"num_write_kb":5548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":3,"acting":3,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":35,"seq":150323855371,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713144,"statfs":{"total":95990841344,"available":95962259456,"internally_reserved":0,"allocated":983040,"data_stored":748221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":5,"up_from":31,"seq":133143986188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713720,"statfs":{"total":95990841344,"available":95962849280,"internally_reserved":0,"allocated":393216,"data_stored":158126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]},{"osd":1,"up_from":20,"seq":85899345935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713724,"statfs":{"total":95990841344,"available":95962853376,"internally_reserved":0,"allocated":389120,"data_stored":157853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]}]},{"osd":0,"up_from":16,"seq":68719476752,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713144,"statfs":{"total":95990841344,"available":95962259456,"internally_reserved":0,"allocated":983040,"data_stored":748221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":2,"up_from":20,"seq":85899345934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713720,"statfs":{"total":95990841344,"available":95962849280,"internally_reserved":0,"allocated":393216,"data_stored":158126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":4,"up_from":24,"seq":103079215117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713720,"statfs":{"total":95990841344,"available":95962849280,"internally_reserved":0,"allocated":393216,"data_stored":158126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182413,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713144,"statfs":{"total":95990841344,"available":95962259456,"internally_reserved":0,"allocated":983040,"data_stored":748171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]}]},{"osd":6,"up_from":29,"seq":124554051596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713720,"statfs":{"total":95990841344,"available":95962849280,"internally_reserved":0,"allocated":393216,"data_stored":158126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-24T09:30:19.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph pg dump --format=json 2023-06-24T09:30:19.984 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:20.296 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:20 smithi045 ceph-mon[102509]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:20.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:20 smithi188 ceph-mon[112326]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:21.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:21 smithi188 ceph-mon[112326]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:21.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:21 smithi045 ceph-mon[102509]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:21.461 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:30:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2023-06-24T09:30:21.928 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2023-06-24T09:30:20.983517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":154,"num_write":291,"num_write_kb":5548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":749928448,"kb_used":220412,"kb_used_data":4796,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":749708036,"statfs":{"total":767926730752,"available":767701028864,"internally_reserved":0,"allocated":4911104,"data_stored":3034970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001933"},"pg_stats":[{"pgid":"1.0","version":"30'237","reported_seq":339,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-06-24T09:29:40.566710+0000","last_change":"2023-06-24T09:29:40.566442+0000","last_active":"2023-06-24T09:29:40.566710+0000","last_peered":"2023-06-24T09:29:40.566710+0000","last_clean":"2023-06-24T09:29:40.566710+0000","last_became_active":"2023-06-24T09:29:40.566054+0000","last_became_peered":"2023-06-24T09:29:40.566054+0000","last_unstale":"2023-06-24T09:29:40.566710+0000","last_undegraded":"2023-06-24T09:29:40.566710+0000","last_fullsized":"2023-06-24T09:29:40.566710+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-24T09:29:21.465010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-24T09:29:21.465010+0000","last_clean_scrub_stamp":"2023-06-24T09:29:21.465010+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T20:46:59.569784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":154,"num_write":291,"num_write_kb":5548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":154,"num_write":291,"num_write_kb":5548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":3,"acting":3,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":35,"seq":150323855371,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713144,"statfs":{"total":95990841344,"available":95962259456,"internally_reserved":0,"allocated":983040,"data_stored":748221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":5,"up_from":31,"seq":133143986188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713720,"statfs":{"total":95990841344,"available":95962849280,"internally_reserved":0,"allocated":393216,"data_stored":158126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]},{"osd":1,"up_from":20,"seq":85899345935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713724,"statfs":{"total":95990841344,"available":95962853376,"internally_reserved":0,"allocated":389120,"data_stored":157853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]}]},{"osd":0,"up_from":16,"seq":68719476752,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713144,"statfs":{"total":95990841344,"available":95962259456,"internally_reserved":0,"allocated":983040,"data_stored":748221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":2,"up_from":20,"seq":85899345934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713720,"statfs":{"total":95990841344,"available":95962849280,"internally_reserved":0,"allocated":393216,"data_stored":158126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":4,"up_from":24,"seq":103079215118,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713720,"statfs":{"total":95990841344,"available":95962849280,"internally_reserved":0,"allocated":393216,"data_stored":158126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182414,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713144,"statfs":{"total":95990841344,"available":95962259456,"internally_reserved":0,"allocated":983040,"data_stored":748171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]}]},{"osd":6,"up_from":29,"seq":124554051597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27336,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93713720,"statfs":{"total":95990841344,"available":95962849280,"internally_reserved":0,"allocated":393216,"data_stored":158126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-24T09:30:21.929 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-06-24T09:30:21.929 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-06-24T09:30:21.929 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-06-24T09:30:21.930 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph health --format=json 2023-06-24T09:30:22.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:22 smithi045 ceph-mon[102509]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:22.172 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:22.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:22 smithi188 ceph-mon[112326]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:23.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:23 smithi045 ceph-mon[102509]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:23.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:30:23.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:23 smithi188 ceph-mon[112326]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:23.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:30:23.630 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:30:23.630 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-06-24T09:30:24.052 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-06-24T09:30:24.053 INFO:tasks.cephadm:Setup complete, yielding 2023-06-24T09:30:24.053 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-06-24T09:30:24.064 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi045.front.sepia.ceph.com 2023-06-24T09:30:24.065 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- bash -c 'ceph orch status' 2023-06-24T09:30:24.305 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:24.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:24 smithi188 ceph-mon[112326]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:24.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:24 smithi188 ceph-mon[112326]: from='client.? 172.21.15.45:0/2824157552' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-06-24T09:30:24.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:24 smithi045 ceph-mon[102509]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:24.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:24 smithi045 ceph-mon[102509]: from='client.? 172.21.15.45:0/2824157552' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-06-24T09:30:25.720 INFO:teuthology.orchestra.run.smithi045.stdout:Backend: cephadm 2023-06-24T09:30:25.721 INFO:teuthology.orchestra.run.smithi045.stdout:Available: Yes 2023-06-24T09:30:25.721 INFO:teuthology.orchestra.run.smithi045.stdout:Paused: No 2023-06-24T09:30:26.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- bash -c 'ceph orch ps' 2023-06-24T09:30:26.364 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:26.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:26 smithi188 ceph-mon[112326]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:26.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:26 smithi045 ceph-mon[102509]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:27.296 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:27 smithi045 ceph-mon[102509]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:27.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:27 smithi188 ceph-mon[112326]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:27.746 INFO:teuthology.orchestra.run.smithi045.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-06-24T09:30:27.747 INFO:teuthology.orchestra.run.smithi045.stdout:alertmanager.smithi045 smithi045 *:9093,9094 running (118s) 49s ago 3m 26.4M - 0.25.0 c8568f914cd2 6c1cff8a0760 2023-06-24T09:30:27.747 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-exporter.smithi045 smithi045 running (3m) 49s ago 3m 7872k - 18.1.0-166-gc9d917e3 8a2463949890 80cf93d5e47a 2023-06-24T09:30:27.747 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-exporter.smithi188 smithi188 running (2m) 51s ago 2m 8212k - 18.1.0-166-gc9d917e3 8a2463949890 018e935ed277 2023-06-24T09:30:27.747 INFO:teuthology.orchestra.run.smithi045.stdout:crash.smithi045 smithi045 running (3m) 49s ago 3m 7432k - 18.1.0-166-gc9d917e3 8a2463949890 f50c859abe0b 2023-06-24T09:30:27.747 INFO:teuthology.orchestra.run.smithi045.stdout:crash.smithi188 smithi188 running (2m) 51s ago 2m 7415k - 18.1.0-166-gc9d917e3 8a2463949890 bb1f6cb1b462 2023-06-24T09:30:27.747 INFO:teuthology.orchestra.run.smithi045.stdout:grafana.smithi045 smithi045 *:3000 running (114s) 49s ago 2m 82.2M - 9.4.7 2c41d148cca3 c48dc0d6d54d 2023-06-24T09:30:27.747 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.smithi045.dcgntd smithi045 *:9283,8765,8443 running (4m) 49s ago 4m 495M - 18.1.0-166-gc9d917e3 8a2463949890 50369365e98d 2023-06-24T09:30:27.747 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.smithi188.zipfzj smithi188 *:8443,9283,8765 running (2m) 51s ago 2m 445M - 18.1.0-166-gc9d917e3 8a2463949890 096dd36e4dbf 2023-06-24T09:30:27.748 INFO:teuthology.orchestra.run.smithi045.stdout:mon.smithi045 smithi045 running (4m) 49s ago 4m 67.2M 2048M 18.1.0-166-gc9d917e3 8a2463949890 a4e89356a2e1 2023-06-24T09:30:27.748 INFO:teuthology.orchestra.run.smithi045.stdout:mon.smithi188 smithi188 running (2m) 51s ago 2m 55.8M 2048M 18.1.0-166-gc9d917e3 8a2463949890 9ba7f01a869d 2023-06-24T09:30:27.748 INFO:teuthology.orchestra.run.smithi045.stdout:node-exporter.smithi045 smithi045 *:9100 running (3m) 49s ago 3m 17.9M - 1.5.0 0da6a335fe13 e7a3f8a03f54 2023-06-24T09:30:27.748 INFO:teuthology.orchestra.run.smithi045.stdout:node-exporter.smithi188 smithi188 *:9100 running (2m) 51s ago 2m 15.4M - 1.5.0 0da6a335fe13 b42e1b8118e7 2023-06-24T09:30:27.748 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 smithi188 running (77s) 51s ago 77s 49.0M 3736M 18.1.0-166-gc9d917e3 8a2463949890 2c5000b4c0ae 2023-06-24T09:30:27.748 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 smithi045 running (73s) 49s ago 72s 52.9M 2969M 18.1.0-166-gc9d917e3 8a2463949890 f78e56456712 2023-06-24T09:30:27.748 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 smithi188 running (73s) 51s ago 73s 41.8M 3736M 18.1.0-166-gc9d917e3 8a2463949890 05afc6503544 2023-06-24T09:30:27.749 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 smithi045 running (68s) 49s ago 67s 45.1M 2969M 18.1.0-166-gc9d917e3 8a2463949890 6290efe8665b 2023-06-24T09:30:27.749 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 smithi188 running (69s) 51s ago 68s 34.9M 3736M 18.1.0-166-gc9d917e3 8a2463949890 2a65002e0768 2023-06-24T09:30:27.749 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 smithi045 running (62s) 49s ago 62s 31.8M 2969M 18.1.0-166-gc9d917e3 8a2463949890 2e32486cd72b 2023-06-24T09:30:27.749 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 smithi188 running (64s) 51s ago 64s 31.6M 3736M 18.1.0-166-gc9d917e3 8a2463949890 b6e3e318e8b0 2023-06-24T09:30:27.749 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 smithi045 running (57s) 49s ago 57s 26.2M 2969M 18.1.0-166-gc9d917e3 8a2463949890 a8c11e517a2b 2023-06-24T09:30:27.749 INFO:teuthology.orchestra.run.smithi045.stdout:prometheus.smithi045 smithi045 *:9095 running (109s) 49s ago 2m 41.3M - 2.43.0 a07b618ecd1d cc4f657fa725 2023-06-24T09:30:28.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- bash -c 'ceph orch ls' 2023-06-24T09:30:28.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:28 smithi188 ceph-mon[112326]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:28.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:28 smithi045 ceph-mon[102509]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:29.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:29 smithi188 ceph-mon[112326]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:29.415 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:29 smithi045 ceph-mon[102509]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:29.841 INFO:teuthology.orchestra.run.smithi045.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-06-24T09:30:29.842 INFO:teuthology.orchestra.run.smithi045.stdout:alertmanager ?:9093,9094 1/1 51s ago 4m count:1 2023-06-24T09:30:29.842 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-exporter 2/2 53s ago 4m * 2023-06-24T09:30:29.842 INFO:teuthology.orchestra.run.smithi045.stdout:crash 2/2 53s ago 4m * 2023-06-24T09:30:29.842 INFO:teuthology.orchestra.run.smithi045.stdout:grafana ?:3000 1/1 51s ago 4m count:1 2023-06-24T09:30:29.842 INFO:teuthology.orchestra.run.smithi045.stdout:mgr 2/2 53s ago 4m count:2 2023-06-24T09:30:29.843 INFO:teuthology.orchestra.run.smithi045.stdout:mon 2/2 53s ago 3m smithi045:172.21.15.45=smithi045;smithi188:172.21.15.188=smithi188;count:2 2023-06-24T09:30:29.843 INFO:teuthology.orchestra.run.smithi045.stdout:node-exporter ?:9100 2/2 53s ago 4m * 2023-06-24T09:30:29.843 INFO:teuthology.orchestra.run.smithi045.stdout:osd.all-available-devices 8 53s ago 2m * 2023-06-24T09:30:29.843 INFO:teuthology.orchestra.run.smithi045.stdout:prometheus ?:9095 1/1 51s ago 4m count:1 2023-06-24T09:30:30.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:30 smithi045 ceph-mon[102509]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:30.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- bash -c 'ceph orch host ls' 2023-06-24T09:30:30.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:30 smithi188 ceph-mon[112326]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:30.526 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:31.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:31 smithi188 ceph-mon[112326]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:31.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:31 smithi045 ceph-mon[102509]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:31.959 INFO:teuthology.orchestra.run.smithi045.stdout:HOST ADDR LABELS STATUS 2023-06-24T09:30:31.959 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 172.21.15.45 2023-06-24T09:30:31.960 INFO:teuthology.orchestra.run.smithi045.stdout:smithi188 172.21.15.188 2023-06-24T09:30:31.960 INFO:teuthology.orchestra.run.smithi045.stdout:2 hosts in cluster 2023-06-24T09:30:32.236 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:32 smithi045 ceph-mon[102509]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:32.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:32 smithi188 ceph-mon[112326]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:32.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- bash -c 'ceph orch device ls' 2023-06-24T09:30:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:33.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:33 smithi045 ceph-mon[102509]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:33.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:33 smithi188 ceph-mon[112326]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:34.127 INFO:teuthology.orchestra.run.smithi045.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-06-24T09:30:34.128 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 51s ago LVM detected, locked 2023-06-24T09:30:34.128 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme1n1 ssd Linux_2434940a0f36a8f91898 89.4G No 51s ago Insufficient space (<10 extents) on vgs, LVM detected, locked 2023-06-24T09:30:34.128 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme2n1 ssd Linux_ea327aca602d1f674717 89.4G No 51s ago Insufficient space (<10 extents) on vgs, LVM detected, locked 2023-06-24T09:30:34.128 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme3n1 ssd Linux_51d783ab236d0aa7745a 89.4G No 51s ago Insufficient space (<10 extents) on vgs, LVM detected, locked 2023-06-24T09:30:34.129 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme4n1 ssd Linux_de742e8002e42c683f9d 89.4G No 51s ago Insufficient space (<10 extents) on vgs, LVM detected, locked 2023-06-24T09:30:34.129 INFO:teuthology.orchestra.run.smithi045.stdout:smithi188 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MP400BGN 372G No 55s ago LVM detected, locked 2023-06-24T09:30:34.129 INFO:teuthology.orchestra.run.smithi045.stdout:smithi188 /dev/nvme1n1 ssd Linux_b42d0a8be2d82d4f8f50 89.4G No 55s ago Insufficient space (<10 extents) on vgs, LVM detected, locked 2023-06-24T09:30:34.130 INFO:teuthology.orchestra.run.smithi045.stdout:smithi188 /dev/nvme2n1 ssd Linux_4f5001aa5e24db854c1a 89.4G No 55s ago Insufficient space (<10 extents) on vgs, LVM detected, locked 2023-06-24T09:30:34.130 INFO:teuthology.orchestra.run.smithi045.stdout:smithi188 /dev/nvme3n1 ssd Linux_36cbbe884b0d8c522544 89.4G No 55s ago Insufficient space (<10 extents) on vgs, LVM detected, locked 2023-06-24T09:30:34.130 INFO:teuthology.orchestra.run.smithi045.stdout:smithi188 /dev/nvme4n1 ssd Linux_57528b3317ca68b9ec0c 89.4G No 55s ago Insufficient space (<10 extents) on vgs, LVM detected, locked 2023-06-24T09:30:34.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:34 smithi188 ceph-mon[112326]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:34.424 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:34 smithi045 ceph-mon[102509]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:34.567 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-06-24T09:30:34.579 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi045.front.sepia.ceph.com 2023-06-24T09:30:34.579 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- bash -c 'ceph orch apply jaeger' 2023-06-24T09:30:34.819 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:35.310 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:35 smithi045 ceph-mon[102509]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:35.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:35 smithi188 ceph-mon[112326]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:36.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:36 smithi045 ceph-mon[102509]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:36.258 INFO:teuthology.orchestra.run.smithi045.stdout:Scheduled elasticsearch update... 2023-06-24T09:30:36.259 INFO:teuthology.orchestra.run.smithi045.stdout:Scheduled jaeger-collector update... 2023-06-24T09:30:36.259 INFO:teuthology.orchestra.run.smithi045.stdout:Scheduled jaeger-query update... 2023-06-24T09:30:36.259 INFO:teuthology.orchestra.run.smithi045.stdout:Scheduled jaeger-agent update... 2023-06-24T09:30:36.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:36 smithi188 ceph-mon[112326]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:36.716 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service... 2023-06-24T09:30:36.725 INFO:tasks.cephadm:Waiting for ceph service elasticsearch to start (timeout 300)... 2023-06-24T09:30:36.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:30:37.333 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:37.582 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:37 smithi045 ceph-mon[102509]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch apply jaeger", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:37.582 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:37 smithi045 ceph-mon[102509]: Saving service elasticsearch spec with placement count:1 2023-06-24T09:30:37.583 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:37.583 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:37 smithi045 ceph-mon[102509]: Saving service jaeger-collector spec with placement count:1 2023-06-24T09:30:37.583 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:37.583 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:37 smithi045 ceph-mon[102509]: Saving service jaeger-query spec with placement count:1 2023-06-24T09:30:37.584 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:37.584 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:37 smithi045 ceph-mon[102509]: Saving service jaeger-agent spec with placement * 2023-06-24T09:30:37.584 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:37.584 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:30:37.585 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:37 smithi045 ceph-mon[102509]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:37.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:37 smithi188 ceph-mon[112326]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch apply jaeger", "target": ["mon-mgr", ""]}]: dispatch 2023-06-24T09:30:37.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:37 smithi188 ceph-mon[112326]: Saving service elasticsearch spec with placement count:1 2023-06-24T09:30:37.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:37.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:37 smithi188 ceph-mon[112326]: Saving service jaeger-collector spec with placement count:1 2023-06-24T09:30:37.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:37.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:37 smithi188 ceph-mon[112326]: Saving service jaeger-query spec with placement count:1 2023-06-24T09:30:37.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:37.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:37 smithi188 ceph-mon[112326]: Saving service jaeger-agent spec with placement * 2023-06-24T09:30:37.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:37.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:30:37.634 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:37 smithi188 ceph-mon[112326]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:39.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:39.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:39.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:30:39.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:39.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:39.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:30:39.213 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:30:39.213 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:29:38.197826Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:29:38.197633Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:29:38.197699Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "ports": [9200], "running": 0, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:29:38.197889Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "ports": [16686], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:29:38.197559Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:29:38.197281Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:29:38.197763Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:29:38.198032Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:29:38.197952Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:30:39.686 INFO:tasks.cephadm:elasticsearch has 0/1 2023-06-24T09:30:39.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:39 smithi045 ceph-mon[102509]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:40.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:39 smithi188 ceph-mon[112326]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:40.687 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:30:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:41.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:41 smithi188 ceph-mon[112326]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:41.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:41 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:41.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:41 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:41.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:41 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:30:41.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:41 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:30:41.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:41 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:41.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:41 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:30:41.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:41 smithi045 ceph-mon[102509]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:41.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:41 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:41.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:41 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:41.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:41 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:30:41.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:41 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:30:41.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:41 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:41.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:41 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:30:42.199 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:42 smithi188 ceph-mon[112326]: Deploying daemon jaeger-agent.smithi188 on smithi188 2023-06-24T09:30:42.200 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:42 smithi188 ceph-mon[112326]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:42.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:42 smithi045 ceph-mon[102509]: Deploying daemon jaeger-agent.smithi188 on smithi188 2023-06-24T09:30:42.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:42 smithi045 ceph-mon[102509]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:42.422 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:30:42.422 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:30:40.069780Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:30:37.765152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:30:37.765293Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "ports": [9200], "running": 0, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:30:40.069844Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "ports": [16686], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:30:37.765435Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:30:37.765499Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:30:37.765368Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:30:37.765562Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:30:40.069906Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:30:42.859 INFO:tasks.cephadm:elasticsearch has 0/1 2023-06-24T09:30:43.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:30:44.134 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:44 smithi045 ceph-mon[102509]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:44 smithi045 ceph-mon[102509]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:44.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:44 smithi188 ceph-mon[112326]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:44.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:44 smithi188 ceph-mon[112326]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:44.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:44.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:44.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:45.249 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:45 smithi045 ceph-mon[102509]: Deploying daemon jaeger-agent.smithi045 on smithi045 2023-06-24T09:30:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:45 smithi188 ceph-mon[112326]: Deploying daemon jaeger-agent.smithi045 on smithi045 2023-06-24T09:30:45.663 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:30:45.663 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:30:40.069780Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:30:37.765152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:30:37.765293Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "ports": [9200], "running": 0, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:30:40.069844Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "ports": [16686], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:30:37.765435Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:30:37.765499Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:30:37.765368Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:30:37.765562Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:30:40.069906Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:30:46.143 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:46 smithi045 ceph-mon[102509]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:46.144 INFO:tasks.cephadm:elasticsearch has 0/1 2023-06-24T09:30:46.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:46 smithi188 ceph-mon[112326]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:47.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:30:47.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:47 smithi188 ceph-mon[112326]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:47.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:47 smithi045 ceph-mon[102509]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:47.770 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:48.354 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:48 smithi045 ceph-mon[102509]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:48.355 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:48 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:48.355 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:48 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:48.355 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:48 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:48.356 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:48 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:48.356 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:48 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:48.379 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:48 smithi188 ceph-mon[112326]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:48.379 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:48 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:48.379 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:48 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:48.380 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:48 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:48.380 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:48 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:48.380 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:48 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:30:49.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:49 smithi188 ceph-mon[112326]: Deploying daemon elasticsearch.smithi188 on smithi188 2023-06-24T09:30:49.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:49 smithi045 ceph-mon[102509]: Deploying daemon elasticsearch.smithi188 on smithi188 2023-06-24T09:30:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:30:49.485 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:30:40.069780Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:30:37.765152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:30:37.765293Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "ports": [9200], "running": 0, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:30:40.069844Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "ports": [16686], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:30:37.765435Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:30:37.765499Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:30:37.765368Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:30:37.765562Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:30:40.069906Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:30:49.958 INFO:tasks.cephadm:elasticsearch has 0/1 2023-06-24T09:30:50.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:50 smithi188 ceph-mon[112326]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:50.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:50 smithi045 ceph-mon[102509]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:50.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:30:51.217 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:51.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:51 smithi188 ceph-mon[112326]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:51.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:51 smithi045 ceph-mon[102509]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:52.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:52 smithi188 ceph-mon[112326]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:52.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:52 smithi045 ceph-mon[102509]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:52.650 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:30:52.650 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:30:40.069780Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:30:37.765152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:30:37.765293Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "ports": [9200], "running": 0, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:30:40.069844Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "ports": [16686], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:30:37.765435Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:30:37.765499Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:30:37.765368Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:30:37.765562Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:30:40.069906Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:30:53.107 INFO:tasks.cephadm:elasticsearch has 0/1 2023-06-24T09:30:53.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:30:53.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:30:54.108 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:30:54.363 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:54 smithi188 ceph-mon[112326]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:54 smithi188 ceph-mon[112326]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:54.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:54 smithi045 ceph-mon[102509]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:54.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:54 smithi045 ceph-mon[102509]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:55.824 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:30:55.824 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:30:40.069780Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:30:37.765152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:30:37.765293Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "ports": [9200], "running": 0, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:30:40.069844Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "ports": [16686], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:30:37.765435Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:30:37.765499Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:30:37.765368Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:30:37.765562Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:30:40.069906Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:30:56.221 INFO:tasks.cephadm:elasticsearch has 0/1 2023-06-24T09:30:56.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:56 smithi188 ceph-mon[112326]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:56.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:56 smithi045 ceph-mon[102509]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:57.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:30:57.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:57 smithi188 ceph-mon[112326]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:57.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:57 smithi045 ceph-mon[102509]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:30:57.480 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:30:58.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:30:58 smithi188 ceph-mon[112326]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:58.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:30:58 smithi045 ceph-mon[102509]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:30:58.963 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:30:58.963 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:30:40.069780Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:30:37.765152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:30:37.765293Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "ports": [9200], "running": 0, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:30:40.069844Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "ports": [16686], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:30:37.765435Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:30:37.765499Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:30:37.765368Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:30:37.765562Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:30:40.069906Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:30:59.556 INFO:tasks.cephadm:elasticsearch has 0/1 2023-06-24T09:31:00.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:00 smithi188 ceph-mon[112326]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:00.136 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:00 smithi188 ceph-mon[112326]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:00.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:00 smithi045 ceph-mon[102509]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:00.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:00 smithi045 ceph-mon[102509]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:00.557 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:00.835 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:01.682 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:01 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:01.683 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:01 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:01.683 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:01 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:01.683 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:01 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:01.683 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:01 smithi045 ceph-mon[102509]: Deploying daemon jaeger-collector.smithi045 on smithi045 2023-06-24T09:31:01.683 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:01 smithi045 ceph-mon[102509]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:01.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:01 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:01.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:01 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:01.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:01 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:01.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:01 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:01.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:01 smithi188 ceph-mon[112326]: Deploying daemon jaeger-collector.smithi045 on smithi045 2023-06-24T09:31:01.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:01 smithi188 ceph-mon[112326]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:02.390 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:02.391 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:30:40.069780Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:30:37.765152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:30:37.765293Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "ports": [9200], "running": 0, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:30:40.069844Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "ports": [16686], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:30:37.765435Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:30:37.765499Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:30:37.765368Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:30:37.765562Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:30:40.069906Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:03.054 INFO:tasks.cephadm:elasticsearch has 0/1 2023-06-24T09:31:04.055 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:04.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:03 smithi188 ceph-mon[112326]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:04.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:03 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:04.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:03 smithi188 ceph-mon[112326]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:04.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:03 smithi045 ceph-mon[102509]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:04.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:03 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:04.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:03 smithi045 ceph-mon[102509]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:04.501 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:05.807 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:05 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:05.808 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:05 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:05.808 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:05 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:05.808 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:05 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:05.808 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:05 smithi188 ceph-mon[112326]: Deploying daemon jaeger-query.smithi188 on smithi188 2023-06-24T09:31:05.808 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:05 smithi188 ceph-mon[112326]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:05.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:05 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:05.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:05 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:05.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:05 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:05.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:05 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:05.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:05 smithi045 ceph-mon[102509]: Deploying daemon jaeger-query.smithi188 on smithi188 2023-06-24T09:31:05.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:05 smithi045 ceph-mon[102509]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:06.029 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:06.029 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:30:40.069780Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:30:37.765152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:30:37.765293Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "ports": [9200], "running": 0, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:30:40.069844Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "ports": [16686], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:30:37.765435Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:30:37.765499Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:30:37.765368Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:30:37.765562Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:30:40.069906Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:06.455 INFO:tasks.cephadm:elasticsearch has 0/1 2023-06-24T09:31:07.018 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:06 smithi188 ceph-mon[112326]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:07.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:06 smithi045 ceph-mon[102509]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:07.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:07.713 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:08.012 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:07 smithi045 ceph-mon[102509]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:08.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:07 smithi188 ceph-mon[112326]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:09.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:09.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:31:09.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:09.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:31:09.227 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:09.227 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:30:40.069780Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:30:37.765152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:30:37.765293Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "ports": [9200], "running": 0, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:30:40.069844Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "ports": [16686], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:30:37.765435Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:30:37.765499Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:30:37.765368Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:30:37.765562Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:30:40.069906Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:09.765 INFO:tasks.cephadm:elasticsearch has 0/1 2023-06-24T09:31:10.104 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:09 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:10.104 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:09 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:10.104 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:09 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:10.104 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:09 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:10.105 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:09 smithi045 ceph-mon[102509]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:10.105 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:09 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:31:10.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:09 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:10.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:09 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:10.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:09 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:10.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:09 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:10.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:09 smithi188 ceph-mon[112326]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:10.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:09 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:31:10.766 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:11.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:10 smithi045 ceph-mon[102509]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:11.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:10 smithi188 ceph-mon[112326]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:11.628 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:12.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:11 smithi188 ceph-mon[112326]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:12.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:11 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:12.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:11 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:12.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:11 smithi045 ceph-mon[102509]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:12.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:11 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:12.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:11 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:13.772 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:13.773 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:30:40.069780Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:30:40.069584Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:30:40.069652Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:11.365551Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:30:40.069844Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:11.365510Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:11.365617Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:30:40.069510Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:30:40.069370Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:30:40.069717Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:30:40.069976Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:30:40.069906Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:14.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:13 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:14.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:13 smithi188 ceph-mon[112326]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:14.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:13 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:14.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:13 smithi045 ceph-mon[102509]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:14.195 INFO:tasks.cephadm:elasticsearch has 1/1 2023-06-24T09:31:14.196 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service... 2023-06-24T09:31:14.205 INFO:tasks.cephadm:Waiting for ceph service jaeger-collector to start (timeout 300)... 2023-06-24T09:31:14.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:14.642 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:15.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:14 smithi045 ceph-mon[102509]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:15.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:14 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:15.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:14 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:15.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:14 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:31:15.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:14 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:31:15.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:14 smithi045 ceph-mon[102509]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:15.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:14 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:15.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:14 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:31:15.175 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:14 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:31:15.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:14 smithi188 ceph-mon[112326]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:15.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:14 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:15.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:14 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:15.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:14 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:31:15.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:14 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:31:15.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:14 smithi188 ceph-mon[112326]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:15.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:14 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:15.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:14 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:31:15.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:14 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:31:16.024 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:15 smithi045 ceph-mon[102509]: Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-06-24T09:31:16.025 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:16.025 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:15 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:16.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:15 smithi188 ceph-mon[112326]: Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2023-06-24T09:31:16.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:16.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:15 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:16.566 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:16.567 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:13.878185Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:13.877973Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:13.878051Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:13.878249Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:13.878631Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:13.878672Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:13.877887Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:13.877721Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:13.878119Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:13.878376Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:13.878312Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:17.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:16 smithi045 ceph-mon[102509]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:17.263 INFO:tasks.cephadm:jaeger-collector has 1/1 2023-06-24T09:31:17.264 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service... 2023-06-24T09:31:17.279 INFO:tasks.cephadm:Waiting for ceph service jaeger-query to start (timeout 300)... 2023-06-24T09:31:17.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:17.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:16 smithi188 ceph-mon[112326]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:18.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:17 smithi045 ceph-mon[102509]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:18.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:17 smithi188 ceph-mon[112326]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:19.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:18 smithi045 ceph-mon[102509]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:19.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:19.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:19.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:31:19.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:31:19.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:19.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:18 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:31:19.327 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:19.328 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:19.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:18 smithi188 ceph-mon[112326]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:19.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:19.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:19.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:31:19.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:31:19.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:19.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:18 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:31:19.868 INFO:tasks.cephadm:jaeger-query has 1/1 2023-06-24T09:31:19.870 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service... 2023-06-24T09:31:19.885 INFO:tasks.cephadm:Waiting for ceph service jaeger-agent to start (timeout 300)... 2023-06-24T09:31:19.886 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:20.132 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:20.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:19 smithi045 ceph-mon[102509]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:20.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:19 smithi045 ceph-mon[102509]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:20.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:19 smithi188 ceph-mon[112326]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:20.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:19 smithi188 ceph-mon[112326]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:21.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:20 smithi045 ceph-mon[102509]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:21.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:20 smithi188 ceph-mon[112326]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:21.631 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:21.631 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:22.087 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:22.169 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:21 smithi188 ceph-mon[112326]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:22.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:21 smithi045 ceph-mon[102509]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:23.089 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:23.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:22 smithi045 ceph-mon[102509]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:23.334 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:23.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:22 smithi188 ceph-mon[112326]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:24.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:24 smithi188 ceph-mon[112326]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:24.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:24.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:31:24.389 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:24 smithi045 ceph-mon[102509]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:24.390 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:31:24.390 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:31:24.803 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:24.804 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:25.242 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:26.243 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:26.277 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:26 smithi045 ceph-mon[102509]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:26.278 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:26 smithi045 ceph-mon[102509]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:26.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:26 smithi188 ceph-mon[112326]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:26.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:26 smithi188 ceph-mon[112326]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:26.501 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:27.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:27 smithi188 ceph-mon[112326]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:27.925 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:27 smithi045 ceph-mon[102509]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:27.974 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:27.975 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:28.392 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:28.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:28 smithi045 ceph-mon[102509]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:28.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:28 smithi188 ceph-mon[112326]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:29.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:29.652 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:29.819 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:29 smithi045 ceph-mon[102509]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:29.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:29 smithi188 ceph-mon[112326]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:31.016 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:31.394 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:31.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:31 smithi045 ceph-mon[102509]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:31.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:31 smithi045 ceph-mon[102509]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:32.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:31 smithi188 ceph-mon[112326]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:32.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:31 smithi188 ceph-mon[112326]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:32.396 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:33.752 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:33 smithi045 ceph-mon[102509]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:34.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:33 smithi188 ceph-mon[112326]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:34.370 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:34.370 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:35.448 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:35.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:35 smithi045 ceph-mon[102509]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:35.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:35 smithi045 ceph-mon[102509]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:36.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:35 smithi188 ceph-mon[112326]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:36.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:35 smithi188 ceph-mon[112326]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:36.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:37.764 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:37 smithi045 ceph-mon[102509]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:37.767 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:37 smithi188 ceph-mon[112326]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:38.155 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:38.155 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:38.558 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:38.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:31:38.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:38 smithi188 ceph-mon[112326]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:38.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:31:38.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:38 smithi045 ceph-mon[102509]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:39.560 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:39.820 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:39.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:39 smithi188 ceph-mon[112326]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:39.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:39 smithi045 ceph-mon[102509]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:41.260 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:41.707 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:41.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:41 smithi045 ceph-mon[102509]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:42.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:41 smithi188 ceph-mon[112326]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:42.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:42.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:42 smithi045 ceph-mon[102509]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:42.969 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:43.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:42 smithi188 ceph-mon[112326]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:43.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:43 smithi045 ceph-mon[102509]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:44.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:43 smithi188 ceph-mon[112326]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:44.465 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:44.890 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:45.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:45.920 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:45 smithi045 ceph-mon[102509]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:45.920 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:45 smithi045 ceph-mon[102509]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:46.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:45 smithi188 ceph-mon[112326]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:46.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:45 smithi188 ceph-mon[112326]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:46.528 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:47.861 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:47 smithi188 ceph-mon[112326]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:47.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:47 smithi045 ceph-mon[102509]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:47.985 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:47.985 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:48.372 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:48.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:48 smithi045 ceph-mon[102509]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:48.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:48 smithi188 ceph-mon[112326]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:49.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:49.636 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:49.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:49 smithi045 ceph-mon[102509]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:49.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:49 smithi188 ceph-mon[112326]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:51.148 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:51.148 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:51.566 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:51.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:51 smithi045 ceph-mon[102509]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:51.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:51 smithi045 ceph-mon[102509]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:52.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:51 smithi188 ceph-mon[112326]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:52.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:51 smithi188 ceph-mon[112326]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:52.568 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:52.826 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:53.893 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:53 smithi045 ceph-mon[102509]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:53.893 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:31:54.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:53 smithi188 ceph-mon[112326]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:54.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:31:54.281 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:54.281 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:54.797 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:55.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:55.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:55 smithi188 ceph-mon[112326]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:55.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:55 smithi188 ceph-mon[112326]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:55.924 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:55 smithi045 ceph-mon[102509]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:55.925 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:55 smithi045 ceph-mon[102509]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:57.489 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:31:57.490 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:31:57.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:57 smithi045 ceph-mon[102509]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:57.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:57 smithi188 ceph-mon[112326]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:58.257 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:31:58.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:58 smithi188 ceph-mon[112326]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:58.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:58 smithi045 ceph-mon[102509]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:31:59.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:31:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:31:59.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:31:59 smithi045 ceph-mon[102509]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:31:59.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:31:59 smithi188 ceph-mon[112326]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:01.000 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:01.001 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:01.380 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:01.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:01 smithi045 ceph-mon[102509]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:01.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:01 smithi045 ceph-mon[102509]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:02.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:01 smithi188 ceph-mon[112326]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:02.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:01 smithi188 ceph-mon[112326]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:02.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:03.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:03 smithi045 ceph-mon[102509]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:04.046 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:04.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:03 smithi188 ceph-mon[112326]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:04.437 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:04.679 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:04 smithi045 ceph-mon[102509]: from='client.24441 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:05.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:04 smithi188 ceph-mon[112326]: from='client.24441 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:05.438 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:05.698 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:05.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:05 smithi045 ceph-mon[102509]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:06.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:05 smithi188 ceph-mon[112326]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:07.146 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:07.146 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:07.569 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:07.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:07 smithi188 ceph-mon[112326]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:07.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:07 smithi188 ceph-mon[112326]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:07.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:07 smithi045 ceph-mon[102509]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:07.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:07 smithi045 ceph-mon[102509]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:08.572 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:08.831 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:08.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:32:08.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:32:09.815 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:09 smithi045 ceph-mon[102509]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:09.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:09 smithi188 ceph-mon[112326]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:10.218 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:10.218 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:10.594 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:11.595 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:11.854 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:11.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:11 smithi045 ceph-mon[102509]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:11.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:11 smithi045 ceph-mon[102509]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:12.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:11 smithi188 ceph-mon[112326]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:12.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:11 smithi188 ceph-mon[112326]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:13.254 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:13.254 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:13.678 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:13.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:13 smithi045 ceph-mon[102509]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:14.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:13 smithi188 ceph-mon[112326]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:14.682 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:14.925 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:14 smithi045 ceph-mon[102509]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:14.940 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:15.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:14 smithi188 ceph-mon[112326]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:15.898 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:15 smithi045 ceph-mon[102509]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:16.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:15 smithi188 ceph-mon[112326]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:16.314 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:16.315 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:16.721 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:17.722 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:17.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:17 smithi188 ceph-mon[112326]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:17.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:17 smithi188 ceph-mon[112326]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:17.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:17 smithi045 ceph-mon[102509]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:17.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:17 smithi045 ceph-mon[102509]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:17.978 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:19.394 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:19.394 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:31:18.563177Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:31:15.705896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:31:15.706091Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:31:15.706698Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:31:18.563240Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:31:15.706655Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:31:18.563676Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:31:15.706767Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:31:15.706242Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:31:15.706312Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:31:15.706170Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:31:15.706380Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:31:18.563302Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:19.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:19 smithi045 ceph-mon[102509]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:19.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:19 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:32:19.917 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:20.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:19 smithi188 ceph-mon[112326]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:20.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:19 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:32:20.919 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:20.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:20 smithi045 ceph-mon[102509]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:20.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:32:20.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:20 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:32:21.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:20 smithi188 ceph-mon[112326]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:21.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:32:21.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:20 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:32:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:21.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:21 smithi045 ceph-mon[102509]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:22.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:21 smithi188 ceph-mon[112326]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:23.014 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:23.493 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:23.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:23 smithi188 ceph-mon[112326]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:23.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:32:23.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:32:23.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:23 smithi188 ceph-mon[112326]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:23.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:32:23.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:32:23.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:32:23.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:23 smithi188 ceph-mon[112326]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:23.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:23 smithi188 ceph-mon[112326]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:23.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:32:23.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:32:23.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:23 smithi045 ceph-mon[102509]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:23.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:32:23.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:32:23.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:23 smithi045 ceph-mon[102509]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:23.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:32:23.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:32:23.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:32:23.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:23 smithi045 ceph-mon[102509]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:23.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:23 smithi045 ceph-mon[102509]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:23.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:32:23.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:32:24.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:26.160 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:26.161 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:26.364 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:26 smithi045 ceph-mon[102509]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:26.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:26 smithi188 ceph-mon[112326]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:26.568 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:27.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:27 smithi188 ceph-mon[112326]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:27.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:27 smithi045 ceph-mon[102509]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:27.569 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:28.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:28 smithi188 ceph-mon[112326]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:28.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:28 smithi045 ceph-mon[102509]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:29.275 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:29.275 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:29.677 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:30.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:30 smithi188 ceph-mon[112326]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:30.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:30 smithi045 ceph-mon[102509]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:30.679 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:30.937 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:31.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:31 smithi188 ceph-mon[112326]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:31.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:31 smithi045 ceph-mon[102509]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:32.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:32 smithi045 ceph-mon[102509]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:32.323 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:32.323 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:32.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:32 smithi188 ceph-mon[112326]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:32.784 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:33.785 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:34.042 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:34.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:34 smithi188 ceph-mon[112326]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:34.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:34 smithi188 ceph-mon[112326]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:34.394 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:34 smithi045 ceph-mon[102509]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:34.395 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:34 smithi045 ceph-mon[102509]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:35.499 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:35.499 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:35.882 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:36.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:36 smithi045 ceph-mon[102509]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:36.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:36 smithi188 ceph-mon[112326]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:36.883 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:37.143 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:37.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:37 smithi045 ceph-mon[102509]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:37.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:37 smithi188 ceph-mon[112326]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:38.363 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:38 smithi045 ceph-mon[102509]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:38.364 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:32:38.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:38 smithi188 ceph-mon[112326]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:38.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:32:38.585 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:38.585 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:38.999 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:40.000 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:40.260 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:40.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:40 smithi188 ceph-mon[112326]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:40.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:40 smithi188 ceph-mon[112326]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:40.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:40 smithi045 ceph-mon[102509]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:40.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:40 smithi045 ceph-mon[102509]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:41.696 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:41.697 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:42.124 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:42.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:42 smithi188 ceph-mon[112326]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:42.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:42 smithi045 ceph-mon[102509]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:43.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:43.372 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:43 smithi188 ceph-mon[112326]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:43.381 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:43.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:43 smithi045 ceph-mon[102509]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:44.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:44 smithi188 ceph-mon[112326]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:44.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:44 smithi045 ceph-mon[102509]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:44.817 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:44.817 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:45.205 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:46.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:46.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:46 smithi188 ceph-mon[112326]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:46.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:46 smithi188 ceph-mon[112326]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:46 smithi045 ceph-mon[102509]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:46 smithi045 ceph-mon[102509]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:46.464 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:47.875 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:47.875 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:47.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:47 smithi188 ceph-mon[112326]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:47.885 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:47 smithi045 ceph-mon[102509]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:48.276 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:48.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:48 smithi188 ceph-mon[112326]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:48.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:48 smithi045 ceph-mon[102509]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:49.277 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:49.536 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:49.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:49 smithi045 ceph-mon[102509]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:49.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:49 smithi188 ceph-mon[112326]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:50.891 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:51.288 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:52.289 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:52.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:52 smithi188 ceph-mon[112326]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:52 smithi188 ceph-mon[112326]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:52.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:52 smithi045 ceph-mon[102509]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:52.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:52 smithi045 ceph-mon[102509]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:52.548 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:53.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:32:53.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:32:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:53.983 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:54 smithi188 ceph-mon[112326]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:54.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:54 smithi045 ceph-mon[102509]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:54.405 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:55.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:55 smithi188 ceph-mon[112326]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:55.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:55.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:55 smithi045 ceph-mon[102509]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:55.664 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:32:56.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:56 smithi045 ceph-mon[102509]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:56.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:56 smithi188 ceph-mon[112326]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:57.018 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:32:57.018 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:32:57.437 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:32:57.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:57 smithi045 ceph-mon[102509]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:57.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:32:57 smithi045 ceph-mon[102509]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:57.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:57 smithi188 ceph-mon[112326]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:32:57.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:32:57 smithi188 ceph-mon[112326]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:32:58.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:32:58.697 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:00.115 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:00.115 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:00.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:00 smithi188 ceph-mon[112326]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:00.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:00 smithi045 ceph-mon[102509]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:00.555 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:01.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:01 smithi188 ceph-mon[112326]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:01.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:01 smithi045 ceph-mon[102509]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:01.557 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:01.818 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:02.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:02 smithi188 ceph-mon[112326]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:02.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:02 smithi045 ceph-mon[102509]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:03.321 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:03.321 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:03.762 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:04.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:04 smithi188 ceph-mon[112326]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:04.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:04 smithi045 ceph-mon[102509]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:04.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:05.022 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:05.366 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:05 smithi045 ceph-mon[102509]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:05.387 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:05 smithi188 ceph-mon[112326]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:06.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:06 smithi188 ceph-mon[112326]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:06.405 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:06.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:06 smithi045 ceph-mon[102509]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:06.794 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:07.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:07.832 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:07 smithi045 ceph-mon[102509]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:07.833 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:07 smithi045 ceph-mon[102509]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:07.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:07 smithi188 ceph-mon[112326]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:07.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:07 smithi188 ceph-mon[112326]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:08.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:33:08.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:33:09.486 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:09.486 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:09.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:09 smithi188 ceph-mon[112326]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:09.883 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:09 smithi045 ceph-mon[102509]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:09.902 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:10.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:10 smithi188 ceph-mon[112326]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:10.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:10.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:10 smithi045 ceph-mon[102509]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:11.164 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:11.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:11 smithi045 ceph-mon[102509]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:11.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:11 smithi188 ceph-mon[112326]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:12.594 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:13.061 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:14.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:14.324 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:14.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:14 smithi188 ceph-mon[112326]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:14.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:14 smithi188 ceph-mon[112326]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:14.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:14 smithi045 ceph-mon[102509]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:14.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:14 smithi045 ceph-mon[102509]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:15.673 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:15.674 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:16.076 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:16.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:16 smithi188 ceph-mon[112326]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:16.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:16 smithi045 ceph-mon[102509]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:17.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:17.337 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:17.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:17 smithi188 ceph-mon[112326]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:17.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:17 smithi045 ceph-mon[102509]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:18.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:18 smithi188 ceph-mon[112326]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:18.389 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:18 smithi045 ceph-mon[102509]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:18.791 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:19.235 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:20.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:20.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:20 smithi188 ceph-mon[112326]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:20.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:20 smithi188 ceph-mon[112326]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:20.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:20 smithi045 ceph-mon[102509]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:20.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:20 smithi045 ceph-mon[102509]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:20.495 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:22.016 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:22.017 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:20.444904Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:20.445065Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:32:20.445646Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:20.445607Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:32:20.445710Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:20.445207Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:20.445271Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:20.445140Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:20.445349Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:22.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:22 smithi188 ceph-mon[112326]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:22.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:22 smithi045 ceph-mon[102509]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:22.459 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:23.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:23 smithi188 ceph-mon[112326]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:23.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:33:23.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:23 smithi045 ceph-mon[102509]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:23.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:33:23.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:24.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:24 smithi188 ceph-mon[112326]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:24.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:24 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:33:24.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:24 smithi045 ceph-mon[102509]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:24.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:24 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:33:25.670 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:32:22.602572Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:32:22.602376Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:32:22.602444Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:32:22.602640Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:32:22.603036Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:32:22.603076Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:32:22.602302Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:32:22.602139Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:32:22.602509Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:32:22.602767Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:32:22.602704Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:26.206 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:25 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:33:26.207 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:25 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:33:26.207 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:25 smithi045 ceph-mon[102509]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:26.208 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:26.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:25 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:33:26.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:25 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:33:26.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:25 smithi188 ceph-mon[112326]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:27.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:27.237 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:26 smithi045 ceph-mon[102509]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:27.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:26 smithi188 ceph-mon[112326]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:27.462 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:28.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:27 smithi045 ceph-mon[102509]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:28.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:27 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:33:28.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:27 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:33:28.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:27 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:33:28.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:27 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:33:28.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:27 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:33:28.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:27 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:33:28.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:27 smithi188 ceph-mon[112326]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:28.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:27 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:33:28.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:27 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:33:28.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:27 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:33:28.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:27 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:33:28.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:27 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:33:28.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:27 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:33:29.034 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:29.034 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:29.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:28 smithi188 ceph-mon[112326]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:29.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:28 smithi188 ceph-mon[112326]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:29.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:28 smithi045 ceph-mon[102509]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:29.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:28 smithi045 ceph-mon[102509]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:29.441 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:30.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:29 smithi188 ceph-mon[112326]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:30.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:29 smithi045 ceph-mon[102509]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:30.442 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:30.700 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:31.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:30 smithi188 ceph-mon[112326]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:31.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:30 smithi045 ceph-mon[102509]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:32.219 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:32.220 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:32.680 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:32.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:32 smithi188 ceph-mon[112326]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:32.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:32 smithi045 ceph-mon[102509]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:33.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:33 smithi045 ceph-mon[102509]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:33.682 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:33.847 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:33 smithi188 ceph-mon[112326]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:34.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:34 smithi188 ceph-mon[112326]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:34.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:34 smithi045 ceph-mon[102509]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:35.398 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:35.399 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:35.814 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:36.816 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:37.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:37 smithi045 ceph-mon[102509]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:37.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:37 smithi045 ceph-mon[102509]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:37.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:37 smithi188 ceph-mon[112326]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:37.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:37 smithi188 ceph-mon[112326]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:38.518 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:38.519 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:38.960 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:39.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:39 smithi188 ceph-mon[112326]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:39.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:39 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:33:39.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:39 smithi045 ceph-mon[102509]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:39.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:39 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:33:39.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:40.221 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:40.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:40 smithi188 ceph-mon[112326]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:40.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:40 smithi045 ceph-mon[102509]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:41.218 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:41 smithi045 ceph-mon[102509]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:41.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:41 smithi188 ceph-mon[112326]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:41.633 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:41.634 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:42.033 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:42.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:42 smithi188 ceph-mon[112326]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:42.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:42 smithi188 ceph-mon[112326]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:42.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:42 smithi045 ceph-mon[102509]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:42.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:42 smithi045 ceph-mon[102509]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:43.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:43.291 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:44.742 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:44.742 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:45.111 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:45 smithi045 ceph-mon[102509]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:45.130 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:45 smithi188 ceph-mon[112326]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:46.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:46.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:46 smithi188 ceph-mon[112326]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:46.389 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:46 smithi045 ceph-mon[102509]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:47.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:47 smithi188 ceph-mon[112326]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:47.424 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:47 smithi045 ceph-mon[102509]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:47.840 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:47.841 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:48.338 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:49.339 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:49.374 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:49 smithi045 ceph-mon[102509]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:49.374 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:49 smithi045 ceph-mon[102509]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:49.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:49 smithi188 ceph-mon[112326]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:49.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:49 smithi188 ceph-mon[112326]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:49.598 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:51.083 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:51.084 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:51.289 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:51 smithi045 ceph-mon[102509]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:51.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:51 smithi188 ceph-mon[112326]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:51.547 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:52.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:52 smithi188 ceph-mon[112326]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:52.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:52 smithi045 ceph-mon[102509]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:52.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:53.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:53 smithi188 ceph-mon[112326]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:53.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:53 smithi045 ceph-mon[102509]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:54.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:54 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:33:54.248 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:54.249 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:54 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:33:54.704 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:55.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:55 smithi045 ceph-mon[102509]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:55.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:55 smithi188 ceph-mon[112326]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:55.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:55.964 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:56.292 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:56 smithi045 ceph-mon[102509]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:56.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:56 smithi188 ceph-mon[112326]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:57.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:57 smithi188 ceph-mon[112326]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:57.388 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:33:57.388 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:33:57.411 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:57 smithi045 ceph-mon[102509]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:57.795 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:33:58.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:33:59.057 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:33:59.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:59 smithi045 ceph-mon[102509]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:59.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:33:59 smithi045 ceph-mon[102509]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:33:59.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:59 smithi188 ceph-mon[112326]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:33:59.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:33:59 smithi188 ceph-mon[112326]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:00.508 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:00.509 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:00.911 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:01.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:01 smithi045 ceph-mon[102509]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:01.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:01 smithi188 ceph-mon[112326]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:01.912 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:02.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:02 smithi188 ceph-mon[112326]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:02.410 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:02 smithi045 ceph-mon[102509]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:03.273 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:03 smithi045 ceph-mon[102509]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:03.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:03 smithi188 ceph-mon[112326]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:03.667 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:04.112 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:05.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:05.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:05 smithi045 ceph-mon[102509]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:05.310 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:05 smithi045 ceph-mon[102509]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:05.371 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:05.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:05 smithi188 ceph-mon[112326]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:05.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:05 smithi188 ceph-mon[112326]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:06.844 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:06.844 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:07.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:07 smithi045 ceph-mon[102509]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:07.295 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:07.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:07 smithi188 ceph-mon[112326]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:08.296 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:08.332 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:08 smithi045 ceph-mon[102509]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:08.333 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:34:08.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:08 smithi188 ceph-mon[112326]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:08.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:34:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:09.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:09 smithi045 ceph-mon[102509]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:09.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:09 smithi188 ceph-mon[112326]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:09.993 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:09.993 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:10.469 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:11.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:11 smithi188 ceph-mon[112326]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:11.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:11 smithi188 ceph-mon[112326]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:11.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:11 smithi045 ceph-mon[102509]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:11.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:11 smithi045 ceph-mon[102509]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:11.470 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:11.729 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:12.714 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:12 smithi045 ceph-mon[102509]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:12.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:12 smithi188 ceph-mon[112326]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:13.117 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:13.118 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:13.492 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:13.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:13 smithi188 ceph-mon[112326]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:13.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:13 smithi045 ceph-mon[102509]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:14.493 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:14.751 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:14.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:14 smithi188 ceph-mon[112326]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:14.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:14 smithi045 ceph-mon[102509]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:16.225 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:16.226 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:16.659 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:17.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:17 smithi188 ceph-mon[112326]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:17.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:17 smithi045 ceph-mon[102509]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:17.660 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:17.917 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:18.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:18 smithi188 ceph-mon[112326]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:18.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:18 smithi045 ceph-mon[102509]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:19.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:19 smithi188 ceph-mon[112326]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:19.420 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:19.420 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:19.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:19 smithi045 ceph-mon[102509]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:19.866 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:20.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:21.126 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:21.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:21 smithi045 ceph-mon[102509]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:21.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:21 smithi045 ceph-mon[102509]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:21.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:21 smithi188 ceph-mon[112326]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:21.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:21 smithi188 ceph-mon[112326]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:22.475 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:22.880 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:22 smithi045 ceph-mon[102509]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:22.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:22 smithi188 ceph-mon[112326]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:22.905 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:23.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:23 smithi188 ceph-mon[112326]: from='client.14864 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:23.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:34:23.906 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:23.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:23 smithi045 ceph-mon[102509]: from='client.14864 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:23.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:34:24.161 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:24.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:24 smithi188 ceph-mon[112326]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:24.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:24 smithi045 ceph-mon[102509]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:25.614 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:25.614 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:26.041 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:27.043 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:27.301 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:27.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:27 smithi188 ceph-mon[112326]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:27.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:27 smithi188 ceph-mon[112326]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:27.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:27 smithi045 ceph-mon[102509]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:27.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:27 smithi045 ceph-mon[102509]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:28.750 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:28.750 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:33:27.473066Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:33:24.963881Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:33:24.964041Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:33:24.964592Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:33:27.473191Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:33:24.964553Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:33:27.473650Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:33:24.964654Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:33:24.964182Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:33:24.964246Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:33:24.964115Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:33:24.964309Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:33:27.473283Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:29.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:29 smithi045 ceph-mon[102509]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:29.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:29 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:34:29.283 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:29.366 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:29 smithi188 ceph-mon[112326]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:29.367 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:29 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:34:30.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:30.314 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:30 smithi045 ceph-mon[102509]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:30.315 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:30 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:34:30.315 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:30 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:34:30.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:30 smithi188 ceph-mon[112326]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:30.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:30 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:34:30.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:30 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:34:30.925 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:31.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:31 smithi188 ceph-mon[112326]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:31.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:31 smithi045 ceph-mon[102509]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:32.360 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:32.361 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:32.820 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:33.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:33 smithi188 ceph-mon[112326]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:33.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:33 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:34:33.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:33 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:34:33.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:33 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:34:33.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:33 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:34:33.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:33 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:34:33.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:33 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:34:33.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:33 smithi045 ceph-mon[102509]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:33.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:33 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:34:33.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:33 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:34:33.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:33 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:34:33.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:33 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:34:33.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:33 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:34:33.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:33 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:34:33.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:34.081 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:34.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:34 smithi188 ceph-mon[112326]: from='client.14876 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:34.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:34 smithi188 ceph-mon[112326]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:34.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:34 smithi188 ceph-mon[112326]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:34.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:34 smithi045 ceph-mon[102509]: from='client.14876 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:34.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:34 smithi045 ceph-mon[102509]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:34.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:34 smithi045 ceph-mon[102509]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:35.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:35 smithi045 ceph-mon[102509]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:35.459 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:35.459 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:35.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:35 smithi188 ceph-mon[112326]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:35.857 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:36.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:36 smithi188 ceph-mon[112326]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:36.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:36 smithi045 ceph-mon[102509]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:36.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:37.115 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:37.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:37 smithi045 ceph-mon[102509]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:37.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:37 smithi188 ceph-mon[112326]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:38.598 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:38.598 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:38.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:34:38.920 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:34:39.080 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:39.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:39 smithi188 ceph-mon[112326]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:39.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:39 smithi188 ceph-mon[112326]: from='client.14884 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:39.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:39 smithi045 ceph-mon[102509]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:39.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:39 smithi045 ceph-mon[102509]: from='client.14884 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:40.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:40.340 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:41.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:41 smithi188 ceph-mon[112326]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:41.883 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:41.908 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:41 smithi045 ceph-mon[102509]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:42.332 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:42.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:42 smithi188 ceph-mon[112326]: from='client.14888 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:42.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:42 smithi045 ceph-mon[102509]: from='client.14888 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:43.333 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:43.592 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:43.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:43 smithi045 ceph-mon[102509]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:43.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:43 smithi188 ceph-mon[112326]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:45.129 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:45.130 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:45.621 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:45.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:45 smithi188 ceph-mon[112326]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:45.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:45 smithi188 ceph-mon[112326]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:45.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:45 smithi045 ceph-mon[102509]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:45.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:45 smithi045 ceph-mon[102509]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:46.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:46.879 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:47.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:47 smithi188 ceph-mon[112326]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:47.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:47 smithi045 ceph-mon[102509]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:48.343 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:48.343 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:48.811 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:49.812 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:49.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:49 smithi188 ceph-mon[112326]: from='client.14896 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:49.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:49 smithi188 ceph-mon[112326]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:49.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:49 smithi045 ceph-mon[102509]: from='client.14896 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:49.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:49 smithi045 ceph-mon[102509]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:50.070 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:51.547 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:51.548 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:51.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:51 smithi188 ceph-mon[112326]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:51.917 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:51 smithi045 ceph-mon[102509]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:51.936 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:52.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:52 smithi188 ceph-mon[112326]: from='client.14900 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:52.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:52 smithi045 ceph-mon[102509]: from='client.14900 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:52.938 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:53.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:53 smithi188 ceph-mon[112326]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:53.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:34:53.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:53 smithi045 ceph-mon[102509]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:53.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:34:54.689 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:54.689 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:55.120 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:55.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:55 smithi188 ceph-mon[112326]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:55.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:55 smithi188 ceph-mon[112326]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:55.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:55 smithi045 ceph-mon[102509]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:55.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:55 smithi045 ceph-mon[102509]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:56.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:56.377 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:57.841 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:34:57.841 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:34:57.872 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:57 smithi045 ceph-mon[102509]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:57.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:57 smithi188 ceph-mon[112326]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:58.329 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:34:58.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:58 smithi045 ceph-mon[102509]: from='client.14908 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:58.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:58 smithi188 ceph-mon[112326]: from='client.14908 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:34:59.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:34:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:34:59.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:34:59 smithi045 ceph-mon[102509]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:34:59.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:34:59 smithi188 ceph-mon[112326]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:01.129 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:01.579 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:01.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:01 smithi188 ceph-mon[112326]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:01.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:01 smithi188 ceph-mon[112326]: from='client.14912 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:01.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:01 smithi045 ceph-mon[102509]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:01.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:01 smithi045 ceph-mon[102509]: from='client.14912 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:02.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:03.881 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:03 smithi045 ceph-mon[102509]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:03.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:03 smithi188 ceph-mon[112326]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:04.322 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:04.323 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:04.776 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:05.778 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:05.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:05 smithi188 ceph-mon[112326]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:05.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:05 smithi188 ceph-mon[112326]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:05.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:05 smithi045 ceph-mon[102509]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:05.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:05 smithi045 ceph-mon[102509]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:06.038 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:07.505 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:07.505 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:07.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:07 smithi188 ceph-mon[112326]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:07.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:07 smithi045 ceph-mon[102509]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:07.961 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:08.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:08 smithi188 ceph-mon[112326]: from='client.14920 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:08.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:35:08.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:08 smithi045 ceph-mon[102509]: from='client.14920 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:08.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:35:08.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:09.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:09 smithi188 ceph-mon[112326]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:09.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:09 smithi045 ceph-mon[102509]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:10.725 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:10.725 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:11.167 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:11.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:11 smithi188 ceph-mon[112326]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:11.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:11 smithi188 ceph-mon[112326]: from='client.14924 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:11.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:11 smithi045 ceph-mon[102509]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:11.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:11 smithi045 ceph-mon[102509]: from='client.14924 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:12.169 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:12.426 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:13.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:13 smithi188 ceph-mon[112326]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:13.894 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:13.895 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:13.919 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:13 smithi045 ceph-mon[102509]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:14.341 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:14.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:14 smithi188 ceph-mon[112326]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:14.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:14 smithi045 ceph-mon[102509]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:15.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:15.600 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:15.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:15 smithi188 ceph-mon[112326]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:15.888 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:15 smithi045 ceph-mon[102509]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:17.016 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:17.017 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:17.483 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:17.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:17 smithi188 ceph-mon[112326]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:17.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:17 smithi188 ceph-mon[112326]: from='client.14932 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:17.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:17 smithi045 ceph-mon[102509]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:17.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:17 smithi045 ceph-mon[102509]: from='client.14932 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:18.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:18.743 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:19.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:19 smithi188 ceph-mon[112326]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:19.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:19 smithi045 ceph-mon[102509]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:20.103 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:20.104 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:20.628 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:20.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:20 smithi188 ceph-mon[112326]: from='client.14936 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:20.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:20 smithi045 ceph-mon[102509]: from='client.14936 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:21.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:21.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:21 smithi188 ceph-mon[112326]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:21.886 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:21.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:21 smithi045 ceph-mon[102509]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:23.291 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:23.723 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:23.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:23 smithi188 ceph-mon[112326]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:23.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:35:23.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:23 smithi045 ceph-mon[102509]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:23.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:35:24.725 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:24.888 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:24 smithi188 ceph-mon[112326]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:24.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:24 smithi045 ceph-mon[102509]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:24.983 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:25.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:25 smithi045 ceph-mon[102509]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:26.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:25 smithi188 ceph-mon[112326]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:26.432 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:26.900 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:27.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:27 smithi188 ceph-mon[112326]: from='client.14944 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:27.957 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:27 smithi188 ceph-mon[112326]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:27.958 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:27.959 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:27 smithi045 ceph-mon[102509]: from='client.14944 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:27.959 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:27 smithi045 ceph-mon[102509]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:29.636 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:29.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:29 smithi188 ceph-mon[112326]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:29.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:29 smithi045 ceph-mon[102509]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:30.047 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:30.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:30 smithi188 ceph-mon[112326]: from='client.14948 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:30.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:30 smithi045 ceph-mon[102509]: from='client.14948 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:31.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:31.308 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:31.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:31 smithi188 ceph-mon[112326]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:31.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:31 smithi045 ceph-mon[102509]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:32.825 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:32.826 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:29.868223Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:29.868497Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:34:29.869082Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:29.869041Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:34:29.869146Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:29.868642Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:29.868709Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:29.868574Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:29.868775Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:33.264 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:33.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:33 smithi188 ceph-mon[112326]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:33.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:33 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:35:33.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:33 smithi188 ceph-mon[112326]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:33.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:33 smithi045 ceph-mon[102509]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:33.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:33 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:35:33.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:33 smithi045 ceph-mon[102509]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:34.265 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:34.666 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:35.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:35 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:35:35.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:35 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:35:35.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:35 smithi188 ceph-mon[112326]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:35.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:35 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:35:35.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:35 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:35:35.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:35 smithi045 ceph-mon[102509]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:36.664 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:36.665 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:34:32.176919Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:34:32.176728Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:34:32.176793Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:34:32.176995Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:34:32.177365Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:34:32.177404Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:34:32.176656Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:34:32.176509Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:34:32.176856Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:34:32.177120Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:34:32.177059Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:37.189 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:38.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:37 smithi188 ceph-mon[112326]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:38.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:37 smithi188 ceph-mon[112326]: from='client.14956 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:38.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:35:38.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:35:38.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:35:38.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:35:38.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:35:38.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:35:38.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:37 smithi045 ceph-mon[102509]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:38.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:37 smithi045 ceph-mon[102509]: from='client.14956 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:38.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:35:38.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:35:38.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:35:38.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:35:38.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:35:38.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:35:38.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:38.447 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:39.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:38 smithi188 ceph-mon[112326]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:39.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:38 smithi188 ceph-mon[112326]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:39.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:35:39.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:38 smithi045 ceph-mon[102509]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:39.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:38 smithi045 ceph-mon[102509]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:39.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:35:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:40.298 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:41.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:40 smithi188 ceph-mon[112326]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:41.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:40 smithi188 ceph-mon[112326]: from='client.14960 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:41.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:40 smithi045 ceph-mon[102509]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:41.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:40 smithi045 ceph-mon[102509]: from='client.14960 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:41.300 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:41.558 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:42.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:42 smithi045 ceph-mon[102509]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:42.982 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:42.982 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:43.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:42 smithi188 ceph-mon[112326]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:43.473 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:43.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:43 smithi045 ceph-mon[102509]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:44.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:43 smithi188 ceph-mon[112326]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:44.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:44.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:44 smithi045 ceph-mon[102509]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:45.072 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:44 smithi188 ceph-mon[112326]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:46.129 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:46.129 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:46.562 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:46.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:46 smithi045 ceph-mon[102509]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:46.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:46 smithi045 ceph-mon[102509]: from='client.14968 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:47.136 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:46 smithi188 ceph-mon[112326]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:47.136 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:46 smithi188 ceph-mon[112326]: from='client.14968 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:47.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:47.823 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:48.849 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:48 smithi045 ceph-mon[102509]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:49.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:48 smithi188 ceph-mon[112326]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:49.238 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:49.239 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:49.678 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:50.680 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:50.938 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:51.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:50 smithi188 ceph-mon[112326]: from='client.14972 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:51.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:50 smithi188 ceph-mon[112326]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:51.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:50 smithi045 ceph-mon[102509]: from='client.14972 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:51.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:50 smithi045 ceph-mon[102509]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:52.493 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:52.493 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:52.867 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:52 smithi045 ceph-mon[102509]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:52.886 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:53.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:52 smithi188 ceph-mon[112326]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:53.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:54.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:53 smithi188 ceph-mon[112326]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:54.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:35:54.146 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:54.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:53 smithi045 ceph-mon[102509]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:54.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:35:54.976 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:54 smithi045 ceph-mon[102509]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:55.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:54 smithi188 ceph-mon[112326]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:55.664 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:55.664 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:56.145 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:57.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:56 smithi188 ceph-mon[112326]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:57.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:56 smithi188 ceph-mon[112326]: from='client.14980 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:57.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:35:57.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:56 smithi045 ceph-mon[102509]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:57.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:56 smithi045 ceph-mon[102509]: from='client.14980 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:35:57.405 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:35:58.868 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:35:58.869 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:35:58.894 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:58 smithi045 ceph-mon[102509]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:59.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:58 smithi188 ceph-mon[112326]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:35:59.260 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:35:59.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:35:59 smithi045 ceph-mon[102509]: from='client.14984 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:00.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:35:59 smithi188 ceph-mon[112326]: from='client.14984 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:00.261 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:00.522 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:01.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:00 smithi188 ceph-mon[112326]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:01.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:00 smithi045 ceph-mon[102509]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:01.953 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:01.953 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:02.343 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:03.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:02 smithi188 ceph-mon[112326]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:03.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:02 smithi188 ceph-mon[112326]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:03.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:02 smithi045 ceph-mon[102509]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:03.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:02 smithi045 ceph-mon[102509]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:03.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:03.602 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:04.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:04 smithi045 ceph-mon[102509]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:05.108 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:05.108 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:05.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:04 smithi188 ceph-mon[112326]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:05.568 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:05.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:05 smithi045 ceph-mon[102509]: from='client.14992 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:06.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:05 smithi188 ceph-mon[112326]: from='client.14992 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:06.569 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:06.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:06 smithi045 ceph-mon[102509]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:07.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:06 smithi188 ceph-mon[112326]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:08.351 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:08.352 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:08.877 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:09.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:08 smithi188 ceph-mon[112326]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:09.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:36:09.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:08 smithi045 ceph-mon[102509]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:09.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:36:09.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:10.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:09 smithi188 ceph-mon[112326]: from='client.14996 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:10.136 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:10.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:09 smithi045 ceph-mon[102509]: from='client.14996 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:11.129 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:10 smithi045 ceph-mon[102509]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:11.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:10 smithi188 ceph-mon[112326]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:11.558 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:11.967 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:12.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:13.003 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:12 smithi045 ceph-mon[102509]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:13.004 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:12 smithi045 ceph-mon[102509]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:13.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:12 smithi188 ceph-mon[112326]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:13.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:12 smithi188 ceph-mon[112326]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:13.226 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:14.739 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:14.740 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:15.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:14 smithi188 ceph-mon[112326]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:15.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:14 smithi045 ceph-mon[102509]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:15.207 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:16.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:15 smithi188 ceph-mon[112326]: from='client.15004 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:16.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:15 smithi045 ceph-mon[102509]: from='client.15004 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:16.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:16.467 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:17.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:16 smithi188 ceph-mon[112326]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:17.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:16 smithi045 ceph-mon[102509]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:17.933 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:17.933 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:18.405 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:19.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:18 smithi188 ceph-mon[112326]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:19.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:18 smithi188 ceph-mon[112326]: from='client.15008 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:19.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:18 smithi045 ceph-mon[102509]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:19.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:18 smithi045 ceph-mon[102509]: from='client.15008 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:19.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:19.664 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:21.073 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:21.073 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:21.101 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:20 smithi045 ceph-mon[102509]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:21.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:20 smithi188 ceph-mon[112326]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:21.506 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:22.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:21 smithi188 ceph-mon[112326]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:22.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:21 smithi045 ceph-mon[102509]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:22.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:22.766 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:22.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:22 smithi045 ceph-mon[102509]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:23.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:22 smithi188 ceph-mon[112326]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:23.876 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:36:24.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:36:24.275 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:24.275 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:24.715 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:25.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:24 smithi188 ceph-mon[112326]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:25.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:24 smithi045 ceph-mon[102509]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:25.717 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:25.976 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:26.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:25 smithi188 ceph-mon[112326]: from='client.15016 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:26.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:25 smithi045 ceph-mon[102509]: from='client.15016 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:27.035 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:26 smithi045 ceph-mon[102509]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:27.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:26 smithi188 ceph-mon[112326]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:27.424 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:27.857 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:28.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:29.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:28 smithi188 ceph-mon[112326]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:29.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:28 smithi188 ceph-mon[112326]: from='client.15020 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:29.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:28 smithi045 ceph-mon[102509]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:29.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:28 smithi045 ceph-mon[102509]: from='client.15020 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:30.614 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:30.614 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:30.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:30 smithi045 ceph-mon[102509]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:31.032 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:31.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:30 smithi188 ceph-mon[112326]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:32.033 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:32.069 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:31 smithi045 ceph-mon[102509]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:32.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:31 smithi188 ceph-mon[112326]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:32.290 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:32.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:32 smithi045 ceph-mon[102509]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:33.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:32 smithi188 ceph-mon[112326]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:33.752 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:33.753 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:34.212 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:35.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:34 smithi188 ceph-mon[112326]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:35.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:34 smithi188 ceph-mon[112326]: from='client.15028 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:35.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:34 smithi045 ceph-mon[102509]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:35.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:34 smithi045 ceph-mon[102509]: from='client.15028 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:35.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:35.473 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:36.875 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:36.875 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:34.468840Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:34.469007Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:35:34.469612Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:34.469570Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:35:34.469677Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:34.469152Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:34.469218Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:34.469084Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:34.469282Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:37.099 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:36 smithi045 ceph-mon[102509]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:37.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:36 smithi188 ceph-mon[112326]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:37.363 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:38.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:37 smithi188 ceph-mon[112326]: from='client.15032 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:38.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:36:38.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:36:38.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:37 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:36:38.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:37 smithi045 ceph-mon[102509]: from='client.15032 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:38.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:36:38.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:36:38.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:37 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:36:38.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:38.606 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:39.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:38 smithi188 ceph-mon[112326]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:39.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:36:39.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:38 smithi045 ceph-mon[102509]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:39.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:36:40.009 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:40.009 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:40.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:40 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:36:40.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:40 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:36:40.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:40 smithi045 ceph-mon[102509]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:40.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:40 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:36:40.385 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:40 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:36:40.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:40 smithi045 ceph-mon[102509]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:40.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:40 smithi045 ceph-mon[102509]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:40.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:40 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:36:40.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:40 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:36:40.386 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:40 smithi045 ceph-mon[102509]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:40.404 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:40.588 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:40 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:36:40.588 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:40 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:36:40.588 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:40 smithi188 ceph-mon[112326]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:40.588 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:40 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:36:40.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:40 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:36:40.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:40 smithi188 ceph-mon[112326]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:40.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:40 smithi188 ceph-mon[112326]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:40.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:40 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:36:40.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:40 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:36:40.589 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:40 smithi188 ceph-mon[112326]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:41.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:41.666 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:42.782 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:42 smithi045 ceph-mon[102509]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:42.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:42 smithi188 ceph-mon[112326]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:43.181 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:43.182 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:43.570 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:43.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:43 smithi188 ceph-mon[112326]: from='client.15040 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:43.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:43 smithi045 ceph-mon[102509]: from='client.15040 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:44.572 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:44.829 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:44.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:44 smithi188 ceph-mon[112326]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:44.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:44 smithi045 ceph-mon[102509]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:46.288 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:46.288 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:46.796 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:46.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:46 smithi045 ceph-mon[102509]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:47.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:46 smithi188 ceph-mon[112326]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:47.797 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:47.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:47 smithi045 ceph-mon[102509]: from='client.15044 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:48.057 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:48.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:47 smithi188 ceph-mon[112326]: from='client.15044 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:48.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:48 smithi045 ceph-mon[102509]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:49.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:48 smithi188 ceph-mon[112326]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:49.544 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:49.544 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:49.936 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:50.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:50 smithi045 ceph-mon[102509]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:50.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:50 smithi045 ceph-mon[102509]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:50.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:51.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:50 smithi188 ceph-mon[112326]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:51.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:50 smithi188 ceph-mon[112326]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:51.193 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:52.540 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:52.540 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:52.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:52 smithi188 ceph-mon[112326]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:52.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:52 smithi045 ceph-mon[102509]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:52.952 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:53.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:53 smithi188 ceph-mon[112326]: from='client.15052 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:53.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:36:53.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:53 smithi045 ceph-mon[102509]: from='client.15052 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:53.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:36:53.954 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:54.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:54 smithi188 ceph-mon[112326]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:54.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:54 smithi045 ceph-mon[102509]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:55.694 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:55.694 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:56.109 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:56.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:56 smithi045 ceph-mon[102509]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:56.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:56 smithi045 ceph-mon[102509]: from='client.15056 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:57.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:36:57.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:56 smithi188 ceph-mon[112326]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:57.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:56 smithi188 ceph-mon[112326]: from='client.15056 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:36:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:36:58.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:58 smithi045 ceph-mon[102509]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:58.960 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:36:58.961 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:36:59.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:58 smithi188 ceph-mon[112326]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:36:59.377 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:36:59.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:36:59 smithi045 ceph-mon[102509]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:00.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:36:59 smithi188 ceph-mon[112326]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:00.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:00.635 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:00.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:00 smithi045 ceph-mon[102509]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:01.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:00 smithi188 ceph-mon[112326]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:02.164 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:02.164 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:02.542 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:02.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:02 smithi188 ceph-mon[112326]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:02.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:02 smithi188 ceph-mon[112326]: from='client.15064 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:02.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:02 smithi045 ceph-mon[102509]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:02.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:02 smithi045 ceph-mon[102509]: from='client.15064 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:03.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:04.863 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:04 smithi045 ceph-mon[102509]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:05.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:04 smithi188 ceph-mon[112326]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:05.259 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:05.259 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:05.669 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:06.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:06.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:06 smithi045 ceph-mon[102509]: from='client.15068 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:06.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:06 smithi045 ceph-mon[102509]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:06.929 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:07.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:06 smithi188 ceph-mon[112326]: from='client.15068 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:07.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:06 smithi188 ceph-mon[112326]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:08.292 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:08.685 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:08.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:08 smithi045 ceph-mon[102509]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:08.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:37:09.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:08 smithi188 ceph-mon[112326]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:09.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:37:09.687 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:09.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:09 smithi045 ceph-mon[102509]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:10.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:09 smithi188 ceph-mon[112326]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:11.072 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:10 smithi045 ceph-mon[102509]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:11.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:10 smithi188 ceph-mon[112326]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:11.512 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:11.513 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:11.917 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:12.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:12 smithi188 ceph-mon[112326]: from='client.15076 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:12.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:12 smithi188 ceph-mon[112326]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:12.918 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:12.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:12 smithi045 ceph-mon[102509]: from='client.15076 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:12.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:12 smithi045 ceph-mon[102509]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:13.176 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:14.635 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:14.636 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:14.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:14 smithi045 ceph-mon[102509]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:15.065 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:15.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:14 smithi188 ceph-mon[112326]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:15.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:15 smithi045 ceph-mon[102509]: from='client.15080 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:16.066 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:16.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:15 smithi188 ceph-mon[112326]: from='client.15080 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:16.925 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:16 smithi045 ceph-mon[102509]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:17.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:16 smithi188 ceph-mon[112326]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:17.809 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:17.809 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:18.215 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:19.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:18 smithi188 ceph-mon[112326]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:19.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:18 smithi188 ceph-mon[112326]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:19.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:18 smithi045 ceph-mon[102509]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:19.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:18 smithi045 ceph-mon[102509]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:19.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:19.476 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:20.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:20 smithi045 ceph-mon[102509]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:20.962 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:20 smithi188 ceph-mon[112326]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:21.030 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:21.031 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:21.449 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:21.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:21 smithi045 ceph-mon[102509]: from='client.15088 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:22.137 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:21 smithi188 ceph-mon[112326]: from='client.15088 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:22.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:22.710 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:22.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:22 smithi045 ceph-mon[102509]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:23.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:22 smithi188 ceph-mon[112326]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:23.859 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:37:24.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:37:24.291 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:24.291 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:24.680 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:24.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:24 smithi045 ceph-mon[102509]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:25.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:24 smithi188 ceph-mon[112326]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:25.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:25.935 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:26.009 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:25 smithi188 ceph-mon[112326]: from='client.15092 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:26.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:25 smithi045 ceph-mon[102509]: from='client.15092 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:26.992 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:26 smithi045 ceph-mon[102509]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:27.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:26 smithi188 ceph-mon[112326]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:27.393 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:27.848 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:28.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:29.112 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:29.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:28 smithi188 ceph-mon[112326]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:29.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:28 smithi188 ceph-mon[112326]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:29.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:28 smithi045 ceph-mon[102509]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:29.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:28 smithi045 ceph-mon[102509]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:30.592 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:30.592 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:30.765 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:30 smithi045 ceph-mon[102509]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:31.013 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:31.055 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:30 smithi188 ceph-mon[112326]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:32.015 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:32.050 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:31 smithi045 ceph-mon[102509]: from='client.15100 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:32.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:31 smithi188 ceph-mon[112326]: from='client.15100 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:32.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:32 smithi045 ceph-mon[102509]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:33.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:32 smithi188 ceph-mon[112326]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:33.790 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:33.790 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:34.221 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:35.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:34 smithi188 ceph-mon[112326]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:35.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:34 smithi188 ceph-mon[112326]: from='client.15104 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:35.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:34 smithi045 ceph-mon[102509]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:35.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:34 smithi045 ceph-mon[102509]: from='client.15104 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:35.222 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:35.480 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:36.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:36 smithi045 ceph-mon[102509]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:36.986 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:36.987 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:37.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:36 smithi188 ceph-mon[112326]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:37.395 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:38.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:37 smithi188 ceph-mon[112326]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:38.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:37 smithi045 ceph-mon[102509]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:38.396 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:38.658 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:39.025 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:38 smithi045 ceph-mon[102509]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:39.025 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:37:39.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:38 smithi188 ceph-mon[112326]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:39.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:37:40.135 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:40.135 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:36:39.219582Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:36:39.219647Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:40.535 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:40.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:40 smithi045 ceph-mon[102509]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:40.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:40 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:37:40.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:40 smithi045 ceph-mon[102509]: from='client.24717 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:41.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:40 smithi188 ceph-mon[112326]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:41.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:40 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:37:41.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:40 smithi188 ceph-mon[112326]: from='client.24717 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:41.535 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:41.796 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:42.782 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:37:42.782 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:37:42.783 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:42 smithi045 ceph-mon[102509]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:42.783 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:37:42.783 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:37:42.783 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:42 smithi045 ceph-mon[102509]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:42.783 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:42 smithi045 ceph-mon[102509]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:42.783 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:37:42.783 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:37:42.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:37:42.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:37:42.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:42 smithi188 ceph-mon[112326]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:42.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:37:42.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:37:42.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:42 smithi188 ceph-mon[112326]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:42.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:42 smithi188 ceph-mon[112326]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:42.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:37:42.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:37:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:43.199 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:43.674 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:43.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:43 smithi188 ceph-mon[112326]: from='client.15116 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:43.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:43 smithi045 ceph-mon[102509]: from='client.15116 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:44.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:44.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:44 smithi188 ceph-mon[112326]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:44.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:44 smithi045 ceph-mon[102509]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:44.932 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:46.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:46.393 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:46.803 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:47.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:46 smithi045 ceph-mon[102509]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:47.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:46 smithi188 ceph-mon[112326]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:47.804 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:48.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:47 smithi045 ceph-mon[102509]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:48.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:47 smithi188 ceph-mon[112326]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:49.156 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:48 smithi045 ceph-mon[102509]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:49.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:48 smithi188 ceph-mon[112326]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:49.550 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:49.550 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:49.928 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:50.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:51.187 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:51.250 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:50 smithi188 ceph-mon[112326]: from='client.15124 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:51.296 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:50 smithi188 ceph-mon[112326]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:51.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:50 smithi045 ceph-mon[102509]: from='client.15124 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:51.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:50 smithi045 ceph-mon[102509]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:52.672 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:52.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:52 smithi188 ceph-mon[112326]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:52.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:52 smithi045 ceph-mon[102509]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:53.075 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:53.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:53 smithi188 ceph-mon[112326]: from='client.15128 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:53.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:37:53.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:53 smithi045 ceph-mon[102509]: from='client.15128 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:53.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:37:54.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:54.334 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:54.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:54 smithi188 ceph-mon[112326]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:54.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:54 smithi045 ceph-mon[102509]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:55.735 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:55.736 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:56.109 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:57.111 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:37:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:37:57.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:56 smithi188 ceph-mon[112326]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:57.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:56 smithi188 ceph-mon[112326]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:57.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:56 smithi045 ceph-mon[102509]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:37:57.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:56 smithi045 ceph-mon[102509]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:58.818 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:37:58.818 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:37:59.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:58 smithi045 ceph-mon[102509]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:37:59.225 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:37:59.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:58 smithi188 ceph-mon[112326]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:00.226 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:00.260 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:37:59 smithi045 ceph-mon[102509]: from='client.15136 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:00.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:37:59 smithi188 ceph-mon[112326]: from='client.15136 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:01.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:00 smithi045 ceph-mon[102509]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:01.342 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:00 smithi188 ceph-mon[112326]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:01.985 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:01.985 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:02.392 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:02.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:02 smithi045 ceph-mon[102509]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:02.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:02 smithi045 ceph-mon[102509]: from='client.15140 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:02.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:02 smithi188 ceph-mon[112326]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:02.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:02 smithi188 ceph-mon[112326]: from='client.15140 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:03.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:03.654 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:05.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:04 smithi045 ceph-mon[102509]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:05.282 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:05.283 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:05.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:04 smithi188 ceph-mon[112326]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:05.719 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:06.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:06.982 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:07.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:06 smithi045 ceph-mon[102509]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:07.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:06 smithi045 ceph-mon[102509]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:07.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:06 smithi188 ceph-mon[112326]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:07.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:06 smithi188 ceph-mon[112326]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:08.460 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:08.460 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:08.891 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:09.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:08 smithi045 ceph-mon[102509]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:09.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:38:09.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:08 smithi188 ceph-mon[112326]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:09.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:38:09.893 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:10.153 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:10.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:09 smithi188 ceph-mon[112326]: from='client.15148 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:10.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:09 smithi045 ceph-mon[102509]: from='client.15148 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:11.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:10 smithi045 ceph-mon[102509]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:11.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:10 smithi188 ceph-mon[112326]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:11.635 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:11.636 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:12.066 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:12.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:12 smithi188 ceph-mon[112326]: from='client.15152 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:12.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:12 smithi188 ceph-mon[112326]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:12.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:12 smithi045 ceph-mon[102509]: from='client.15152 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:12.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:12 smithi045 ceph-mon[102509]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:13.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:13.323 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:14.767 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:15.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:14 smithi045 ceph-mon[102509]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:15.232 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:15.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:14 smithi188 ceph-mon[112326]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:16.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:15 smithi045 ceph-mon[102509]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:16.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:16.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:15 smithi188 ceph-mon[112326]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:16.491 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:17.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:16 smithi045 ceph-mon[102509]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:17.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:16 smithi188 ceph-mon[112326]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:18.075 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:18.075 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:18.543 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:19.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:18 smithi188 ceph-mon[112326]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:19.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:18 smithi188 ceph-mon[112326]: from='client.15160 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:19.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:18 smithi045 ceph-mon[102509]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:19.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:18 smithi045 ceph-mon[102509]: from='client.15160 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:19.544 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:21.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:20 smithi045 ceph-mon[102509]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:21.342 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:21.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:20 smithi188 ceph-mon[112326]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:21.772 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:22.774 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:22.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:22 smithi188 ceph-mon[112326]: from='client.15164 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:22.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:22 smithi188 ceph-mon[112326]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:22.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:22 smithi045 ceph-mon[102509]: from='client.15164 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:22.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:22 smithi045 ceph-mon[102509]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:23.032 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:23.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:38:23.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:38:24.502 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:24.503 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:24.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:24 smithi188 ceph-mon[112326]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:24.919 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:24 smithi045 ceph-mon[102509]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:24.939 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:25.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:25 smithi188 ceph-mon[112326]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:25.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:25 smithi045 ceph-mon[102509]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:25.940 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:26.204 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:26.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:26 smithi188 ceph-mon[112326]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:26.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:26 smithi045 ceph-mon[102509]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:27.694 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:28.089 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:29.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:29.351 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:29.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:28 smithi188 ceph-mon[112326]: from='client.15172 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:29.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:28 smithi188 ceph-mon[112326]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:29.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:28 smithi045 ceph-mon[102509]: from='client.15172 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:29.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:28 smithi045 ceph-mon[102509]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:30.737 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:30.738 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:31.067 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:30 smithi045 ceph-mon[102509]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:31.129 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:31.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:30 smithi188 ceph-mon[112326]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:32.130 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:32.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:31 smithi188 ceph-mon[112326]: from='client.15176 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:32.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:31 smithi045 ceph-mon[102509]: from='client.15176 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:33.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:32 smithi188 ceph-mon[112326]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:33.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:32 smithi045 ceph-mon[102509]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:33.892 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:33.893 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:34.301 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:35.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:34 smithi045 ceph-mon[102509]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:35.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:34 smithi045 ceph-mon[102509]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:35.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:35.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:34 smithi188 ceph-mon[112326]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:35.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:34 smithi188 ceph-mon[112326]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:35.562 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:37.040 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:37.040 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:37.385 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:36 smithi188 ceph-mon[112326]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:37.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:36 smithi045 ceph-mon[102509]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:37.452 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:38.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:37 smithi188 ceph-mon[112326]: from='client.15184 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:38.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:37 smithi045 ceph-mon[102509]: from='client.15184 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:38.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:38.698 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:39.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:38 smithi045 ceph-mon[102509]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:39.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:38:39.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:38 smithi188 ceph-mon[112326]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:39.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:38:40.093 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:40.094 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:40.475 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:41.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:40 smithi188 ceph-mon[112326]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:41.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:40 smithi188 ceph-mon[112326]: from='client.15188 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:41.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:40 smithi045 ceph-mon[102509]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:41.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:40 smithi045 ceph-mon[102509]: from='client.15188 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:41.476 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:42.740 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:42 smithi045 ceph-mon[102509]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:42.740 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:38:42.741 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:38:42.741 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:42 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:38:42.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:42 smithi188 ceph-mon[112326]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:42.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:38:42.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:38:42.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:42 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:38:43.147 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:43.148 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:43.560 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:43.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:38:43.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:38:43.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:43 smithi188 ceph-mon[112326]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:43.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:43 smithi188 ceph-mon[112326]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:43.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:38:43.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:38:43.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:43 smithi188 ceph-mon[112326]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:43.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:38:43.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:38:43.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:43 smithi045 ceph-mon[102509]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:43.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:43 smithi045 ceph-mon[102509]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:43.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:38:43.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:38:43.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:43 smithi045 ceph-mon[102509]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:44.561 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:44.818 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:45.886 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:45 smithi045 ceph-mon[102509]: pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:46.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:45 smithi188 ceph-mon[112326]: pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:46.269 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:46.270 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:46.719 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:47.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:47.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:47 smithi188 ceph-mon[112326]: from='client.15196 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:47.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:47 smithi188 ceph-mon[112326]: pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:47.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:47 smithi045 ceph-mon[102509]: from='client.15196 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:47.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:47 smithi045 ceph-mon[102509]: pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:47.979 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:49.346 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:49.347 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:49.743 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:50.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:49 smithi188 ceph-mon[112326]: pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:50.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:49 smithi045 ceph-mon[102509]: pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:50.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:51.005 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:51.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:50 smithi188 ceph-mon[112326]: from='client.15200 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:51.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:50 smithi045 ceph-mon[102509]: from='client.15200 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:52.122 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:51 smithi045 ceph-mon[102509]: pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:52.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:51 smithi188 ceph-mon[112326]: pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:52.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:52.531 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:52.935 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:53.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:54.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:53 smithi188 ceph-mon[112326]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:54.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:53 smithi188 ceph-mon[112326]: pgmap v371: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:54.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:38:54.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:53 smithi045 ceph-mon[102509]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:54.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:53 smithi045 ceph-mon[102509]: pgmap v371: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:54.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:38:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:55.734 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:55.734 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:56.131 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:55 smithi045 ceph-mon[102509]: pgmap v372: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:56.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:55 smithi188 ceph-mon[112326]: pgmap v372: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:56.149 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:38:57.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:56 smithi188 ceph-mon[112326]: from='client.15208 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:57.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:38:57.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:56 smithi045 ceph-mon[102509]: from='client.15208 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:38:57.409 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:38:57.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:57 smithi045 ceph-mon[102509]: pgmap v373: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:58.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:57 smithi188 ceph-mon[112326]: pgmap v373: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:38:58.935 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:38:58.936 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:38:59.358 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:00.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:59 smithi188 ceph-mon[112326]: pgmap v374: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:00.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:38:59 smithi188 ceph-mon[112326]: from='client.15212 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:00.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:59 smithi045 ceph-mon[102509]: pgmap v374: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:00.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:38:59 smithi045 ceph-mon[102509]: from='client.15212 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:00.360 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:00.620 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:02.071 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:02.071 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:02.097 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:01 smithi045 ceph-mon[102509]: pgmap v375: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:02.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:01 smithi188 ceph-mon[112326]: pgmap v375: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:02.498 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:03.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:02 smithi188 ceph-mon[112326]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:03.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:02 smithi045 ceph-mon[102509]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:03.500 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:03.758 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:03.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:03 smithi045 ceph-mon[102509]: pgmap v376: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:04.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:03 smithi188 ceph-mon[112326]: pgmap v376: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:05.211 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:05.212 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:05.807 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:05 smithi045 ceph-mon[102509]: pgmap v377: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:05.808 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:05 smithi045 ceph-mon[102509]: from='client.15220 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:05.809 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:06.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:05 smithi188 ceph-mon[112326]: pgmap v377: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:06.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:05 smithi188 ceph-mon[112326]: from='client.15220 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:06.810 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:07.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:07 smithi188 ceph-mon[112326]: pgmap v378: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:07.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:07 smithi045 ceph-mon[102509]: pgmap v378: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:08.542 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:08.542 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:08.884 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:39:08.904 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:39:08.923 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:09.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:09 smithi188 ceph-mon[112326]: from='client.15224 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:09.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:09 smithi188 ceph-mon[112326]: pgmap v379: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:09.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:09 smithi045 ceph-mon[102509]: from='client.15224 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:09.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:09 smithi045 ceph-mon[102509]: pgmap v379: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:09.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:10.183 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:11.587 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:11.587 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:11.963 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:12.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:11 smithi188 ceph-mon[112326]: pgmap v380: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:12.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:11 smithi045 ceph-mon[102509]: pgmap v380: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:12.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:13.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:12 smithi188 ceph-mon[112326]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:13.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:12 smithi045 ceph-mon[102509]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:13.221 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:14.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:13 smithi188 ceph-mon[112326]: pgmap v381: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:14.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:13 smithi045 ceph-mon[102509]: pgmap v381: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:14.740 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:14.741 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:15.139 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:16.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:15 smithi188 ceph-mon[112326]: pgmap v382: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:16.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:15 smithi188 ceph-mon[112326]: from='client.15232 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:16.142 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:16.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:15 smithi045 ceph-mon[102509]: pgmap v382: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:16.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:15 smithi045 ceph-mon[102509]: from='client.15232 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:16.402 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:17.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:17 smithi188 ceph-mon[112326]: pgmap v383: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:17.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:17 smithi045 ceph-mon[102509]: pgmap v383: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:17.934 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:17.934 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:18.313 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:18.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:18 smithi188 ceph-mon[112326]: from='client.15236 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:18.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:18 smithi045 ceph-mon[102509]: from='client.15236 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:19.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:19.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:19 smithi188 ceph-mon[112326]: pgmap v384: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:19.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:19 smithi045 ceph-mon[102509]: pgmap v384: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:21.284 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:21.739 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:22.091 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:21 smithi188 ceph-mon[112326]: pgmap v385: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:22.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:21 smithi045 ceph-mon[102509]: pgmap v385: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:22.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:23.002 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:23.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:22 smithi188 ceph-mon[112326]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:23.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:22 smithi045 ceph-mon[102509]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:24.041 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:23 smithi045 ceph-mon[102509]: pgmap v386: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:24.041 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:39:24.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:23 smithi188 ceph-mon[112326]: pgmap v386: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:24.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:39:24.438 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:24.439 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:24.905 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:25.906 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:26.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:25 smithi188 ceph-mon[112326]: from='client.15244 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:26.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:25 smithi188 ceph-mon[112326]: pgmap v387: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:26.164 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:26.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:25 smithi045 ceph-mon[102509]: from='client.15244 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:26.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:25 smithi045 ceph-mon[102509]: pgmap v387: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:27.657 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:27.657 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:27.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:27 smithi188 ceph-mon[112326]: pgmap v388: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:27.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:27 smithi045 ceph-mon[102509]: pgmap v388: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:28.153 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:28.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:28 smithi188 ceph-mon[112326]: from='client.15248 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:28.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:28 smithi045 ceph-mon[102509]: from='client.15248 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:29.155 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:29.420 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:29.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:29 smithi188 ceph-mon[112326]: pgmap v389: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:29.924 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:29 smithi045 ceph-mon[102509]: pgmap v389: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:30.906 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:30.907 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:31.341 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:31.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:31 smithi045 ceph-mon[102509]: pgmap v390: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:31.923 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:31 smithi045 ceph-mon[102509]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:32.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:31 smithi188 ceph-mon[112326]: pgmap v390: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:32.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:31 smithi188 ceph-mon[112326]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:32.342 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:32.601 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:34.100 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:34.100 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:34.120 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:33 smithi045 ceph-mon[102509]: pgmap v391: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:34.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:33 smithi188 ceph-mon[112326]: pgmap v391: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:34.502 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:35.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:34 smithi188 ceph-mon[112326]: from='client.15256 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:35.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:34 smithi045 ceph-mon[102509]: from='client.15256 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:35.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:36.028 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:35 smithi045 ceph-mon[102509]: pgmap v392: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:36.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:35 smithi188 ceph-mon[112326]: pgmap v392: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:37.386 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:37.386 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:37.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:37 smithi045 ceph-mon[102509]: pgmap v393: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:37.813 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:37.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:37 smithi188 ceph-mon[112326]: pgmap v393: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:38.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:38.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:38 smithi188 ceph-mon[112326]: from='client.15260 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:38.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:39:38.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:38 smithi045 ceph-mon[102509]: from='client.15260 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:38.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:39:39.075 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:39.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:39 smithi045 ceph-mon[102509]: pgmap v394: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:40.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:39 smithi188 ceph-mon[112326]: pgmap v394: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:40.544 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:40.545 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:41.035 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:42.036 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:42.071 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:41 smithi045 ceph-mon[102509]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:42.071 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:41 smithi045 ceph-mon[102509]: pgmap v395: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:42.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:41 smithi188 ceph-mon[112326]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:42.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:41 smithi188 ceph-mon[112326]: pgmap v395: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:42.294 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:43.778 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:43.779 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:44.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: pgmap v396: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:44.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:39:44.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: Adjusting osd_memory_target on smithi188 to 2870M 2023-06-24T09:39:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:39:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:39:44.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:39:44.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:39:44.135 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:39:44.136 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:39:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: pgmap v396: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:39:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: Adjusting osd_memory_target on smithi188 to 2870M 2023-06-24T09:39:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:39:44.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-24T09:39:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:39:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:39:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:39:44.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:39:44.174 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:39:44.259 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:45.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:44 smithi188 ceph-mon[112326]: Adjusting osd_memory_target on smithi045 to 2457M 2023-06-24T09:39:45.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:44 smithi188 ceph-mon[112326]: pgmap v397: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:45.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:44 smithi188 ceph-mon[112326]: pgmap v398: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:45.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:44 smithi188 ceph-mon[112326]: from='client.15268 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:45.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:44 smithi045 ceph-mon[102509]: Adjusting osd_memory_target on smithi045 to 2457M 2023-06-24T09:39:45.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:44 smithi045 ceph-mon[102509]: pgmap v397: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:45.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:44 smithi045 ceph-mon[102509]: pgmap v398: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:45.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:44 smithi045 ceph-mon[102509]: from='client.15268 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:45.260 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:45.519 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:46.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:46 smithi045 ceph-mon[102509]: pgmap v399: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:47.027 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:47.027 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:47.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:46 smithi188 ceph-mon[112326]: pgmap v399: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:47.468 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:48.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:47 smithi188 ceph-mon[112326]: from='client.15272 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:48.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:47 smithi045 ceph-mon[102509]: from='client.15272 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:48.470 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:49.091 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:48 smithi045 ceph-mon[102509]: pgmap v400: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:49.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:48 smithi188 ceph-mon[112326]: pgmap v400: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:50.216 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:50.217 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:50.617 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:50.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:50 smithi045 ceph-mon[102509]: pgmap v401: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:50.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:50 smithi045 ceph-mon[102509]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:51.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:50 smithi188 ceph-mon[112326]: pgmap v401: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:51.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:50 smithi188 ceph-mon[112326]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:51.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:51.880 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:53.004 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:52 smithi045 ceph-mon[102509]: pgmap v402: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:53.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:52 smithi188 ceph-mon[112326]: pgmap v402: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:53.410 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:53.410 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:53.858 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:54.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:39:54.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:39:54.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:55.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:54 smithi188 ceph-mon[112326]: pgmap v403: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:55.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:54 smithi188 ceph-mon[112326]: from='client.15280 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:55.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:54 smithi045 ceph-mon[102509]: pgmap v403: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:55.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:54 smithi045 ceph-mon[102509]: from='client.15280 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:56.616 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:56.616 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:39:56.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:56 smithi045 ceph-mon[102509]: pgmap v404: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:57.084 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:39:57.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:56 smithi188 ceph-mon[112326]: pgmap v404: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:58.086 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:39:58.120 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:57 smithi045 ceph-mon[102509]: from='client.15284 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:58.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:57 smithi188 ceph-mon[112326]: from='client.15284 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:39:58.345 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:39:58.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:39:58 smithi045 ceph-mon[102509]: pgmap v405: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:59.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:39:58 smithi188 ceph-mon[112326]: pgmap v405: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:39:59.825 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:39:59.825 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:00.252 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:01.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:00 smithi188 ceph-mon[112326]: pgmap v406: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:01.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:00 smithi188 ceph-mon[112326]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:01.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:00 smithi188 ceph-mon[112326]: Health detail: HEALTH_WARN 2 failed cephadm daemon(s) 2023-06-24T09:40:01.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:00 smithi188 ceph-mon[112326]: [WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2023-06-24T09:40:01.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:00 smithi188 ceph-mon[112326]: daemon jaeger-agent.smithi045 on smithi045 is in error state 2023-06-24T09:40:01.134 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:00 smithi188 ceph-mon[112326]: daemon jaeger-agent.smithi188 on smithi188 is in error state 2023-06-24T09:40:01.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:00 smithi045 ceph-mon[102509]: pgmap v406: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:01.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:00 smithi045 ceph-mon[102509]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:01.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:00 smithi045 ceph-mon[102509]: Health detail: HEALTH_WARN 2 failed cephadm daemon(s) 2023-06-24T09:40:01.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:00 smithi045 ceph-mon[102509]: [WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2023-06-24T09:40:01.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:00 smithi045 ceph-mon[102509]: daemon jaeger-agent.smithi045 on smithi045 is in error state 2023-06-24T09:40:01.173 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:00 smithi045 ceph-mon[102509]: daemon jaeger-agent.smithi188 on smithi188 is in error state 2023-06-24T09:40:01.253 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:03.114 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:03.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:02 smithi188 ceph-mon[112326]: pgmap v407: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:03.147 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:02 smithi045 ceph-mon[102509]: pgmap v407: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:03.524 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:04.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:03 smithi188 ceph-mon[112326]: from='client.15292 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:04.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:03 smithi045 ceph-mon[102509]: from='client.15292 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:04.526 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:05.080 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:04 smithi045 ceph-mon[102509]: pgmap v408: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:05.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:04 smithi188 ceph-mon[112326]: pgmap v408: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:06.223 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:06.224 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:06.641 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:06.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:06 smithi045 ceph-mon[102509]: pgmap v409: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:06.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:06 smithi045 ceph-mon[102509]: from='client.15296 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:07.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:06 smithi188 ceph-mon[112326]: pgmap v409: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:07.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:06 smithi188 ceph-mon[112326]: from='client.15296 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:07.642 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:07.902 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:08.852 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:08 smithi045 ceph-mon[102509]: pgmap v410: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:08.852 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:40:09.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:08 smithi188 ceph-mon[112326]: pgmap v410: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:09.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:40:09.259 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:09.260 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:09.648 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:10.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:10.906 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:11.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:10 smithi188 ceph-mon[112326]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:11.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:10 smithi188 ceph-mon[112326]: pgmap v411: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:11.140 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:10 smithi045 ceph-mon[102509]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:11.141 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:10 smithi045 ceph-mon[102509]: pgmap v411: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:12.443 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:12.443 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:12.822 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:13.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:12 smithi188 ceph-mon[112326]: pgmap v412: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:13.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:12 smithi045 ceph-mon[102509]: pgmap v412: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:13.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:14.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:13 smithi188 ceph-mon[112326]: from='client.15304 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:14.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:13 smithi045 ceph-mon[102509]: from='client.15304 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:15.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:14 smithi188 ceph-mon[112326]: pgmap v413: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:15.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:14 smithi045 ceph-mon[102509]: pgmap v413: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:15.629 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:15.630 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:16.013 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:17.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:17.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:16 smithi188 ceph-mon[112326]: pgmap v414: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:17.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:16 smithi188 ceph-mon[112326]: from='client.15308 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:17.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:16 smithi045 ceph-mon[102509]: pgmap v414: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:17.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:16 smithi045 ceph-mon[102509]: from='client.15308 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:17.277 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:18.797 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:19.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:18 smithi188 ceph-mon[112326]: pgmap v415: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:19.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:18 smithi045 ceph-mon[102509]: pgmap v415: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:19.195 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:20.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:19 smithi188 ceph-mon[112326]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:20.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:19 smithi045 ceph-mon[102509]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:20.197 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:20.456 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:21.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:20 smithi188 ceph-mon[112326]: pgmap v416: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:21.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:20 smithi045 ceph-mon[102509]: pgmap v416: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:21.948 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:21.949 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:22.522 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:23.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:22 smithi188 ceph-mon[112326]: pgmap v417: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:23.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:22 smithi188 ceph-mon[112326]: from='client.15316 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:23.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:22 smithi045 ceph-mon[102509]: pgmap v417: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:23.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:22 smithi045 ceph-mon[102509]: from='client.15316 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:23.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:24.124 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:40:24.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:40:25.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:24 smithi188 ceph-mon[112326]: pgmap v418: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:25.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:24 smithi045 ceph-mon[102509]: pgmap v418: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:25.214 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:25.215 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:25.620 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:26.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:25 smithi188 ceph-mon[112326]: from='client.15320 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:26.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:25 smithi045 ceph-mon[102509]: from='client.15320 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:26.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:26.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:26 smithi045 ceph-mon[102509]: pgmap v419: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:27.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:26 smithi188 ceph-mon[112326]: pgmap v419: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:28.428 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:28.429 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:28.920 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:29.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:28 smithi188 ceph-mon[112326]: pgmap v420: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:29.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:28 smithi045 ceph-mon[102509]: pgmap v420: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:29.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:30.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:29 smithi188 ceph-mon[112326]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:30.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:29 smithi045 ceph-mon[102509]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:30.908 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:30 smithi045 ceph-mon[102509]: pgmap v421: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:31.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:30 smithi188 ceph-mon[112326]: pgmap v421: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:31.620 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:31.621 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:32.053 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:33.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:33.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:32 smithi188 ceph-mon[112326]: pgmap v422: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:33.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:32 smithi188 ceph-mon[112326]: from='client.15328 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:33.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:32 smithi045 ceph-mon[102509]: pgmap v422: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:33.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:32 smithi045 ceph-mon[102509]: from='client.15328 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:34.846 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:34.847 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:35.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:34 smithi188 ceph-mon[112326]: pgmap v423: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:35.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:34 smithi045 ceph-mon[102509]: pgmap v423: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:35.242 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:36.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:35 smithi188 ceph-mon[112326]: from='client.15332 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:36.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:35 smithi045 ceph-mon[102509]: from='client.15332 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:36.243 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:36.503 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:37.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:36 smithi045 ceph-mon[102509]: pgmap v424: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:37.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:36 smithi188 ceph-mon[112326]: pgmap v424: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:37.994 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:37.994 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:38.418 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:39.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:38 smithi045 ceph-mon[102509]: pgmap v425: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:39.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:38 smithi045 ceph-mon[102509]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:39.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:40:39.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:38 smithi188 ceph-mon[112326]: pgmap v425: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:39.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:38 smithi188 ceph-mon[112326]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:39.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:40:39.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:39.678 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:41.170 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:41.170 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:41.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:40 smithi045 ceph-mon[102509]: pgmap v426: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:41.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:40 smithi188 ceph-mon[112326]: pgmap v426: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:41.569 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:42.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:41 smithi045 ceph-mon[102509]: from='client.15340 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:42.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:41 smithi188 ceph-mon[112326]: from='client.15340 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:42.570 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:42.829 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:43.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:42 smithi188 ceph-mon[112326]: pgmap v427: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:43.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:42 smithi045 ceph-mon[102509]: pgmap v427: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:43.898 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:40:43.898 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:40:43.899 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:43 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:40:44.308 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:44.309 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:44.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:40:44.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:40:44.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:43 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:40:44.700 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:45.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:44 smithi045 ceph-mon[102509]: pgmap v428: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:45.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:44 smithi045 ceph-mon[102509]: pgmap v429: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:45.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:44 smithi045 ceph-mon[102509]: pgmap v430: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:45.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:40:45.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:40:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:44 smithi188 ceph-mon[112326]: pgmap v428: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:44 smithi188 ceph-mon[112326]: pgmap v429: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:44 smithi188 ceph-mon[112326]: pgmap v430: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:40:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:40:45.702 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:45.960 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:46.162 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:45 smithi045 ceph-mon[102509]: from='client.15344 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:46.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:45 smithi188 ceph-mon[112326]: from='client.15344 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:47.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:46 smithi045 ceph-mon[102509]: pgmap v431: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:47.334 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:47.334 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:47.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:46 smithi188 ceph-mon[112326]: pgmap v431: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:47.751 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:48.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:49.010 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:49.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:48 smithi045 ceph-mon[102509]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:49.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:48 smithi045 ceph-mon[102509]: pgmap v432: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:49.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:48 smithi188 ceph-mon[112326]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:49.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:48 smithi188 ceph-mon[112326]: pgmap v432: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:50.485 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:50.908 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:51.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:50 smithi045 ceph-mon[102509]: pgmap v433: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:51.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:50 smithi188 ceph-mon[112326]: pgmap v433: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:51.908 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:52.167 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:52.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:51 smithi045 ceph-mon[102509]: from='client.15352 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:52.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:51 smithi188 ceph-mon[112326]: from='client.15352 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:53.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:52 smithi045 ceph-mon[102509]: pgmap v434: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:53.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:52 smithi188 ceph-mon[112326]: pgmap v434: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:53.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:53.652 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:54.029 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:40:54.047 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:40:55.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:55.306 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:55.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:54 smithi045 ceph-mon[102509]: from='client.15356 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:55.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:54 smithi045 ceph-mon[102509]: pgmap v435: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:55.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:54 smithi188 ceph-mon[112326]: from='client.15356 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:55.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:54 smithi188 ceph-mon[112326]: pgmap v435: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:56.866 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:40:56.866 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:40:57.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:56 smithi045 ceph-mon[102509]: pgmap v436: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:57.299 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:40:57.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:56 smithi188 ceph-mon[112326]: pgmap v436: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:58.300 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:40:58.335 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:57 smithi045 ceph-mon[102509]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:58.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:57 smithi188 ceph-mon[112326]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:40:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:40:59.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:40:58 smithi045 ceph-mon[102509]: pgmap v437: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:40:59.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:40:58 smithi188 ceph-mon[112326]: pgmap v437: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:00.051 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:00.442 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:01.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:00 smithi188 ceph-mon[112326]: pgmap v438: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:01.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:00 smithi188 ceph-mon[112326]: from='client.15364 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:01.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:00 smithi045 ceph-mon[102509]: pgmap v438: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:01.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:00 smithi045 ceph-mon[102509]: from='client.15364 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:01.444 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:01.705 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:02.793 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:02 smithi045 ceph-mon[102509]: pgmap v439: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:03.029 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:02 smithi188 ceph-mon[112326]: pgmap v439: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:03.196 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:03.647 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:03.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:03 smithi045 ceph-mon[102509]: from='client.15368 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:04.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:03 smithi188 ceph-mon[112326]: from='client.15368 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:04.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:04.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:04 smithi045 ceph-mon[102509]: pgmap v440: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:05.138 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:04 smithi188 ceph-mon[112326]: pgmap v440: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:06.317 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:06.318 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:06.723 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:07.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:06 smithi045 ceph-mon[102509]: pgmap v441: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:07.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:06 smithi188 ceph-mon[112326]: pgmap v441: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:07.724 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:08.233 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:07 smithi045 ceph-mon[102509]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:08.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:07 smithi188 ceph-mon[112326]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:09.060 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:08 smithi045 ceph-mon[102509]: pgmap v442: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:09.060 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:41:09.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:08 smithi188 ceph-mon[112326]: pgmap v442: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:09.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:41:09.474 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:09.474 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:09.881 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:10.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:11.140 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:11.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:10 smithi045 ceph-mon[102509]: from='client.15376 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:11.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:10 smithi045 ceph-mon[102509]: pgmap v443: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:11.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:10 smithi188 ceph-mon[112326]: from='client.15376 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:11.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:10 smithi188 ceph-mon[112326]: pgmap v443: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:12.627 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:12.627 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:12.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:12 smithi045 ceph-mon[102509]: pgmap v444: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:13.048 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:13.123 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:12 smithi188 ceph-mon[112326]: pgmap v444: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:13.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:13 smithi045 ceph-mon[102509]: from='client.15380 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:14.050 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:14.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:13 smithi188 ceph-mon[112326]: from='client.15380 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:14.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:14 smithi045 ceph-mon[102509]: pgmap v445: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:15.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:14 smithi188 ceph-mon[112326]: pgmap v445: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:15.849 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:15.850 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:16.237 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:17.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:17.275 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:16 smithi045 ceph-mon[102509]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:17.275 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:16 smithi045 ceph-mon[102509]: pgmap v446: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:17.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:16 smithi188 ceph-mon[112326]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:17.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:16 smithi188 ceph-mon[112326]: pgmap v446: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:19.017 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:19.018 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:19.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:18 smithi188 ceph-mon[112326]: pgmap v447: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:19.409 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:18 smithi045 ceph-mon[102509]: pgmap v447: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:19.429 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:20.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:19 smithi188 ceph-mon[112326]: from='client.15388 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:20.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:19 smithi045 ceph-mon[102509]: from='client.15388 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:20.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:20.689 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:21.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:20 smithi045 ceph-mon[102509]: pgmap v448: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:21.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:20 smithi188 ceph-mon[112326]: pgmap v448: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:22.084 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:22.084 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:22.486 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:22.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:22 smithi045 ceph-mon[102509]: pgmap v449: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:22.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:22 smithi045 ceph-mon[102509]: from='client.15392 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:23.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:22 smithi188 ceph-mon[112326]: pgmap v449: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:23.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:22 smithi188 ceph-mon[112326]: from='client.15392 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:23.487 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:23.746 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:23.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:41:24.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:41:24.839 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:24 smithi045 ceph-mon[102509]: pgmap v450: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:25.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:24 smithi188 ceph-mon[112326]: pgmap v450: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:25.278 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:25.278 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:25.726 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:26.727 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:26.985 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:27.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:26 smithi045 ceph-mon[102509]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:27.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:26 smithi045 ceph-mon[102509]: pgmap v451: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:27.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:26 smithi188 ceph-mon[112326]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:27.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:26 smithi188 ceph-mon[112326]: pgmap v451: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:28.476 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:28.476 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:28.877 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:29.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:28 smithi045 ceph-mon[102509]: pgmap v452: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:29.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:28 smithi188 ceph-mon[112326]: pgmap v452: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:29.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:30.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:29 smithi045 ceph-mon[102509]: from='client.15400 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:30.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:29 smithi188 ceph-mon[112326]: from='client.15400 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:31.257 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:30 smithi045 ceph-mon[102509]: pgmap v453: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:31.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:30 smithi188 ceph-mon[112326]: pgmap v453: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:31.653 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:31.653 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:32.044 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:32.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:32 smithi045 ceph-mon[102509]: from='client.15404 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:32.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:32 smithi045 ceph-mon[102509]: pgmap v454: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:33.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:33.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:32 smithi188 ceph-mon[112326]: from='client.15404 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:33.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:32 smithi188 ceph-mon[112326]: pgmap v454: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:34.677 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:34.678 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:35.147 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:35.310 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:34 smithi045 ceph-mon[102509]: pgmap v455: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:35.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:34 smithi188 ceph-mon[112326]: pgmap v455: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:36.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:36.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:35 smithi188 ceph-mon[112326]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:36.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:35 smithi045 ceph-mon[102509]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:37.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:36 smithi188 ceph-mon[112326]: pgmap v456: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:37.396 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:36 smithi045 ceph-mon[102509]: pgmap v456: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:37.803 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:37.803 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:38.225 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:39.226 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:39.261 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:38 smithi045 ceph-mon[102509]: from='client.15412 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:39.261 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:38 smithi045 ceph-mon[102509]: pgmap v457: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:39.262 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:41:39.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:38 smithi188 ceph-mon[112326]: from='client.15412 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:39.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:38 smithi188 ceph-mon[112326]: pgmap v457: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:39.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:41:39.486 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:40.974 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:40.974 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:41.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:40 smithi188 ceph-mon[112326]: pgmap v458: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:41.391 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:40 smithi045 ceph-mon[102509]: pgmap v458: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:41.410 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:42.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:41 smithi188 ceph-mon[112326]: from='client.15416 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:42.412 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:42.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:41 smithi045 ceph-mon[102509]: from='client.15416 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:43.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:42 smithi045 ceph-mon[102509]: pgmap v459: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:43.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:42 smithi188 ceph-mon[112326]: pgmap v459: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:44.204 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:44.204 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:44.627 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:45.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:44 smithi045 ceph-mon[102509]: pgmap v460: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:45.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:41:45.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:44 smithi045 ceph-mon[102509]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:45.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:41:45.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:41:45.310 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:41:45.310 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:44 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:41:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:44 smithi188 ceph-mon[112326]: pgmap v460: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:41:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:44 smithi188 ceph-mon[112326]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:41:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:41:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:41:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:44 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:41:45.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:45.883 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:46.240 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:45 smithi045 ceph-mon[102509]: pgmap v461: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:46.241 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:45 smithi045 ceph-mon[102509]: pgmap v462: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:46.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:45 smithi188 ceph-mon[112326]: pgmap v461: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:46.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:45 smithi188 ceph-mon[112326]: pgmap v462: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:47.428 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:47.428 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:47.820 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:48.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:47 smithi045 ceph-mon[102509]: pgmap v463: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:48.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:47 smithi188 ceph-mon[112326]: pgmap v463: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:48.821 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:49.079 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:49.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:48 smithi045 ceph-mon[102509]: from='client.15424 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:49.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:48 smithi188 ceph-mon[112326]: from='client.15424 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:50.228 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:49 smithi045 ceph-mon[102509]: pgmap v464: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:50.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:49 smithi188 ceph-mon[112326]: pgmap v464: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:50.667 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:50.668 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:51.065 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:52.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:52.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:51 smithi188 ceph-mon[112326]: pgmap v465: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:52.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:51 smithi188 ceph-mon[112326]: from='client.15428 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:52.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:51 smithi045 ceph-mon[102509]: pgmap v465: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:52.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:51 smithi045 ceph-mon[102509]: from='client.15428 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:53.901 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:54.280 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:53 smithi045 ceph-mon[102509]: pgmap v466: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:54.280 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:41:54.299 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:53 smithi188 ceph-mon[112326]: pgmap v466: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:41:55.300 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:55.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:54 smithi045 ceph-mon[102509]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:55.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:54 smithi188 ceph-mon[112326]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:41:56.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:55 smithi045 ceph-mon[102509]: pgmap v467: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:56.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:55 smithi188 ceph-mon[112326]: pgmap v467: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:57.088 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:41:57.088 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:41:57.498 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:41:58.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:58 smithi188 ceph-mon[112326]: pgmap v468: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:58.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:41:58 smithi188 ceph-mon[112326]: from='client.15436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:58.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:58 smithi045 ceph-mon[102509]: pgmap v468: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:41:58.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:41:58 smithi045 ceph-mon[102509]: from='client.15436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:41:58.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:41:58.759 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:00.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:00 smithi045 ceph-mon[102509]: pgmap v469: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:00.210 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:00.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:00 smithi188 ceph-mon[112326]: pgmap v469: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:00.622 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:01.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:01 smithi188 ceph-mon[112326]: from='client.15440 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:01.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:01 smithi045 ceph-mon[102509]: from='client.15440 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:01.623 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:02.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:02 smithi188 ceph-mon[112326]: pgmap v470: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:02.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:02 smithi045 ceph-mon[102509]: pgmap v470: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:03.380 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:03.380 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:03.780 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:04.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:04 smithi045 ceph-mon[102509]: pgmap v471: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:04.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:04 smithi188 ceph-mon[112326]: pgmap v471: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:04.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:05.038 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:05.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:05 smithi045 ceph-mon[102509]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:05.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:05 smithi188 ceph-mon[112326]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:06.104 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:06 smithi045 ceph-mon[102509]: pgmap v472: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:06.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:06 smithi188 ceph-mon[112326]: pgmap v472: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:06.495 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:06.495 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:06.955 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:07.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:08.219 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:08.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:08 smithi188 ceph-mon[112326]: pgmap v473: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:08.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:08 smithi188 ceph-mon[112326]: from='client.15448 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:08.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:08 smithi045 ceph-mon[102509]: pgmap v473: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:08.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:08 smithi045 ceph-mon[102509]: from='client.15448 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:09.233 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:09 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:42:09.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:09 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:42:09.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:09.644 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:10.063 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:10.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:10 smithi188 ceph-mon[112326]: pgmap v474: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:10.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:10 smithi045 ceph-mon[102509]: pgmap v474: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:11.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:11.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:11 smithi188 ceph-mon[112326]: from='client.15452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:11.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:11 smithi045 ceph-mon[102509]: from='client.15452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:12.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:12 smithi188 ceph-mon[112326]: pgmap v475: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:12.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:12 smithi045 ceph-mon[102509]: pgmap v475: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:12.814 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:12.814 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:13.244 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:14.245 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:14.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:14 smithi188 ceph-mon[112326]: pgmap v476: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:14.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:14 smithi188 ceph-mon[112326]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:14.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:14 smithi045 ceph-mon[102509]: pgmap v476: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:14.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:14 smithi045 ceph-mon[102509]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:16.000 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:16.001 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:16.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:16 smithi188 ceph-mon[112326]: pgmap v477: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:16.392 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:16 smithi045 ceph-mon[102509]: pgmap v477: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:16.410 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:17.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:17 smithi188 ceph-mon[112326]: from='client.15460 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:17.412 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:17.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:17 smithi045 ceph-mon[102509]: from='client.15460 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:18.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:18 smithi045 ceph-mon[102509]: pgmap v478: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:18.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:18 smithi188 ceph-mon[112326]: pgmap v478: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:19.177 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:19.178 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:19.595 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:20.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:20 smithi188 ceph-mon[112326]: pgmap v479: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:20.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:20 smithi188 ceph-mon[112326]: from='client.15464 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:20.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:20 smithi045 ceph-mon[102509]: pgmap v479: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:20.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:20 smithi045 ceph-mon[102509]: from='client.15464 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:20.596 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:20.854 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:22.337 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:22.361 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:22 smithi045 ceph-mon[102509]: pgmap v480: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:22.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:22 smithi188 ceph-mon[112326]: pgmap v480: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:22.786 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:23.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:42:23.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:42:23.788 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:24.045 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:24.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:24 smithi045 ceph-mon[102509]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:24.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:24 smithi045 ceph-mon[102509]: pgmap v481: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:24.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:24 smithi188 ceph-mon[112326]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:24.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:24 smithi188 ceph-mon[112326]: pgmap v481: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:25.599 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:25.599 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:26.011 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:26.257 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:26 smithi045 ceph-mon[102509]: pgmap v482: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:26.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:26 smithi188 ceph-mon[112326]: pgmap v482: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:27.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:27.272 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:27.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:27 smithi188 ceph-mon[112326]: from='client.15472 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:27.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:27 smithi045 ceph-mon[102509]: from='client.15472 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:28.356 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:28 smithi045 ceph-mon[102509]: pgmap v483: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:28.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:28 smithi188 ceph-mon[112326]: pgmap v483: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:28.753 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:28.753 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:29.185 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:30.186 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:30.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:30 smithi188 ceph-mon[112326]: pgmap v484: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:30.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:30 smithi188 ceph-mon[112326]: from='client.15476 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:30.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:30 smithi045 ceph-mon[102509]: pgmap v484: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:30.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:30 smithi045 ceph-mon[102509]: from='client.15476 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:31.862 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:31.862 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:32.170 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:32 smithi045 ceph-mon[102509]: pgmap v485: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:32.259 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:32.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:32 smithi188 ceph-mon[112326]: pgmap v485: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:33.261 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:33.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:33 smithi188 ceph-mon[112326]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:33.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:33 smithi045 ceph-mon[102509]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:33.523 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:34.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:34 smithi188 ceph-mon[112326]: pgmap v486: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:34.348 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:34 smithi045 ceph-mon[102509]: pgmap v486: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:35.045 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:35.045 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:35.450 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:36.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:36 smithi188 ceph-mon[112326]: pgmap v487: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:36.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:36 smithi188 ceph-mon[112326]: from='client.15484 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:36.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:36 smithi045 ceph-mon[102509]: pgmap v487: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:36.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:36 smithi045 ceph-mon[102509]: from='client.15484 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:36.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:36.710 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:38.258 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:38.258 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:38.361 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:38 smithi045 ceph-mon[102509]: pgmap v488: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:38.361 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:42:38.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:38 smithi188 ceph-mon[112326]: pgmap v488: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:38.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:42:38.651 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:39.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:39 smithi188 ceph-mon[112326]: from='client.15488 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:39.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:39 smithi045 ceph-mon[102509]: from='client.15488 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:39.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:39.912 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:40.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:40 smithi188 ceph-mon[112326]: pgmap v489: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:40.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:40 smithi045 ceph-mon[102509]: pgmap v489: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:41.416 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:41.809 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:42.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:42 smithi188 ceph-mon[112326]: pgmap v490: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:42.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:42 smithi045 ceph-mon[102509]: pgmap v490: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:42.810 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:43.067 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:43.344 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:43 smithi045 ceph-mon[102509]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:43.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:43 smithi188 ceph-mon[112326]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:44.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:44 smithi045 ceph-mon[102509]: pgmap v491: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:44.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:44 smithi188 ceph-mon[112326]: pgmap v491: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:44.602 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:44.602 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:45.015 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:45.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:42:45.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:42:45.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:42:45.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:42:45.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:45 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:42:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:42:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:42:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:42:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:42:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:45 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:42:46.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:46.272 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:46.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:46 smithi188 ceph-mon[112326]: pgmap v492: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:46.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:46 smithi188 ceph-mon[112326]: from='client.15496 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:46.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:46 smithi188 ceph-mon[112326]: pgmap v493: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:46.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:46 smithi188 ceph-mon[112326]: pgmap v494: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:46 smithi045 ceph-mon[102509]: pgmap v492: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:46 smithi045 ceph-mon[102509]: from='client.15496 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:46 smithi045 ceph-mon[102509]: pgmap v493: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:46 smithi045 ceph-mon[102509]: pgmap v494: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:47.772 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:47.773 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:47.950 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:47 smithi045 ceph-mon[102509]: pgmap v495: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:48.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:47 smithi188 ceph-mon[112326]: pgmap v495: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:48.165 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:48.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:48 smithi045 ceph-mon[102509]: from='client.15500 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:49.011 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:48 smithi188 ceph-mon[112326]: from='client.15500 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:49.166 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:49.422 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:50.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:49 smithi188 ceph-mon[112326]: pgmap v496: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:50.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:49 smithi045 ceph-mon[102509]: pgmap v496: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:50.957 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:51.356 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:52.356 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:52.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:52 smithi188 ceph-mon[112326]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:52 smithi188 ceph-mon[112326]: pgmap v497: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:52.392 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:52 smithi045 ceph-mon[102509]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:52.393 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:52 smithi045 ceph-mon[102509]: pgmap v497: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:52.615 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:53.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:42:53.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:42:54.211 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:54.211 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:54 smithi188 ceph-mon[112326]: pgmap v498: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:54.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:54 smithi045 ceph-mon[102509]: pgmap v498: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:54.613 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:55.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:55 smithi188 ceph-mon[112326]: from='client.15508 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:55.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:55 smithi045 ceph-mon[102509]: from='client.15508 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:55.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:55.872 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:56.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:56 smithi188 ceph-mon[112326]: pgmap v499: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:56.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:56 smithi045 ceph-mon[102509]: pgmap v499: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:57.413 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:42:57.413 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:42:57.802 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:57 smithi045 ceph-mon[102509]: pgmap v500: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:57.822 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:42:58.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:57 smithi188 ceph-mon[112326]: pgmap v500: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:42:58.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:42:58.857 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:58 smithi045 ceph-mon[102509]: from='client.15512 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:42:59.082 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:42:59.106 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:58 smithi188 ceph-mon[112326]: from='client.15512 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:00.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:42:59 smithi188 ceph-mon[112326]: pgmap v501: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:00.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:42:59 smithi045 ceph-mon[102509]: pgmap v501: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:00.578 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:01.006 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:02.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:02.269 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:02.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:02 smithi188 ceph-mon[112326]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:02.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:02 smithi188 ceph-mon[112326]: pgmap v502: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:02.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:02 smithi045 ceph-mon[102509]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:02.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:02 smithi045 ceph-mon[102509]: pgmap v502: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:03.789 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:03.789 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:04.239 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:04.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:04 smithi188 ceph-mon[112326]: pgmap v503: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:04.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:04 smithi045 ceph-mon[102509]: pgmap v503: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:05.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:05.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:05 smithi188 ceph-mon[112326]: from='client.15520 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:05.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:05 smithi045 ceph-mon[102509]: from='client.15520 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:06.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:06 smithi188 ceph-mon[112326]: pgmap v504: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:06.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:06 smithi045 ceph-mon[102509]: pgmap v504: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:07.027 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:07.027 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:07.421 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:08.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:07 smithi188 ceph-mon[112326]: from='client.15524 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:08.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:07 smithi188 ceph-mon[112326]: pgmap v505: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:08.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:07 smithi045 ceph-mon[102509]: from='client.15524 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:08.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:07 smithi045 ceph-mon[102509]: pgmap v505: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:08.422 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:08.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:43:09.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:43:09.791 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:09 smithi045 ceph-mon[102509]: pgmap v506: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:10.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:09 smithi188 ceph-mon[112326]: pgmap v506: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:10.218 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:10.218 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:10.612 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:10.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:10 smithi045 ceph-mon[102509]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:11.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:10 smithi188 ceph-mon[112326]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:11.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:11.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:11 smithi045 ceph-mon[102509]: pgmap v507: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:12.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:11 smithi188 ceph-mon[112326]: pgmap v507: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:13.389 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:13.389 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:13.819 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:14.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:14 smithi188 ceph-mon[112326]: pgmap v508: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:14.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:14 smithi045 ceph-mon[102509]: pgmap v508: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:14.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:15.377 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:15 smithi045 ceph-mon[102509]: from='client.15532 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:15.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:15 smithi188 ceph-mon[112326]: from='client.15532 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:16.271 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:16 smithi045 ceph-mon[102509]: pgmap v509: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:16.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:16 smithi188 ceph-mon[112326]: pgmap v509: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:16.690 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:16.691 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:17.160 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:18.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:17 smithi188 ceph-mon[112326]: from='client.15536 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:18.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:17 smithi188 ceph-mon[112326]: pgmap v510: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:18.161 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:18.170 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:17 smithi045 ceph-mon[102509]: from='client.15536 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:18.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:17 smithi045 ceph-mon[102509]: pgmap v510: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:19.917 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:19.917 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:20.341 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:20 smithi045 ceph-mon[102509]: pgmap v511: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:20.360 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:20.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:20 smithi188 ceph-mon[112326]: pgmap v511: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:21.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:21.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:21 smithi188 ceph-mon[112326]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:21.396 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:21 smithi045 ceph-mon[102509]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:22.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:22 smithi045 ceph-mon[102509]: pgmap v512: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:22.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:22 smithi188 ceph-mon[112326]: pgmap v512: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:23.039 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:23.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:43:23.383 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:43:23.424 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:24.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:24 smithi188 ceph-mon[112326]: from='client.15544 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:24.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:24 smithi188 ceph-mon[112326]: pgmap v513: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:24.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:24 smithi045 ceph-mon[102509]: from='client.15544 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:24.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:24 smithi045 ceph-mon[102509]: pgmap v513: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:24.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:24.685 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:26.254 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:26.255 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:26.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:26 smithi188 ceph-mon[112326]: pgmap v514: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:26.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:26 smithi045 ceph-mon[102509]: pgmap v514: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:26.785 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:27.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:27 smithi188 ceph-mon[112326]: from='client.15548 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:27.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:27 smithi045 ceph-mon[102509]: from='client.15548 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:27.786 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:28.044 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:28.372 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:28 smithi045 ceph-mon[102509]: pgmap v515: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:28.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:28 smithi188 ceph-mon[112326]: pgmap v515: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:29.440 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:29.441 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:29.826 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:30.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:30 smithi188 ceph-mon[112326]: pgmap v516: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:30.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:30 smithi045 ceph-mon[102509]: pgmap v516: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:30.828 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:31.085 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:31.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:31 smithi188 ceph-mon[112326]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:31.396 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:31 smithi045 ceph-mon[102509]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:32.212 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:32 smithi045 ceph-mon[102509]: pgmap v517: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:32.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:32 smithi188 ceph-mon[112326]: pgmap v517: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:32.633 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:32.633 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:33.035 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:34.036 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:34.290 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:34.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:34 smithi045 ceph-mon[102509]: from='client.15556 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:34.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:34 smithi045 ceph-mon[102509]: pgmap v518: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:34.434 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:34 smithi188 ceph-mon[112326]: from='client.15556 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:34.435 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:34 smithi188 ceph-mon[112326]: pgmap v518: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:35.778 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:35.778 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:36.208 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:36.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:36 smithi045 ceph-mon[102509]: pgmap v519: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:36.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:36 smithi188 ceph-mon[112326]: pgmap v519: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:37.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:37.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:37 smithi045 ceph-mon[102509]: from='client.15560 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:37.468 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:37.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:37 smithi188 ceph-mon[112326]: from='client.15560 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:38.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:38 smithi045 ceph-mon[102509]: pgmap v520: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:38.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:43:38.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:38 smithi188 ceph-mon[112326]: pgmap v520: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:38.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:43:38.968 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:38.968 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:39.418 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:40.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:40.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:40 smithi045 ceph-mon[102509]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:40.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:40 smithi045 ceph-mon[102509]: pgmap v521: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:40.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:40 smithi188 ceph-mon[112326]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:40.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:40 smithi188 ceph-mon[112326]: pgmap v521: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:42.151 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:42.151 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:42.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:42 smithi045 ceph-mon[102509]: pgmap v522: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:42.608 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:42.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:42 smithi188 ceph-mon[112326]: pgmap v522: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:43.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:43 smithi045 ceph-mon[102509]: from='client.15568 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:43.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:43.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:43 smithi188 ceph-mon[112326]: from='client.15568 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:43.866 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:44.254 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:44 smithi045 ceph-mon[102509]: pgmap v523: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:44.528 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:44 smithi188 ceph-mon[112326]: pgmap v523: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:45.448 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:45.449 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:45.890 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:46 smithi045 ceph-mon[102509]: pgmap v524: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:46 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:43:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:46 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:43:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:46 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:43:46.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:46 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:43:46.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:46 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:43:46.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:46 smithi188 ceph-mon[112326]: pgmap v524: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:46.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:46 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:43:46.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:46 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:43:46.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:46 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:43:46.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:46 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:43:46.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:46 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:43:46.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:47.399 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:47 smithi045 ceph-mon[102509]: from='client.15572 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:47.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:47 smithi045 ceph-mon[102509]: pgmap v525: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:47.400 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:47 smithi045 ceph-mon[102509]: pgmap v526: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:47.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:47 smithi188 ceph-mon[112326]: from='client.15572 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:47.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:47 smithi188 ceph-mon[112326]: pgmap v525: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:47.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:47 smithi188 ceph-mon[112326]: pgmap v526: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:48.670 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:48.671 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:49.167 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:49.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:49 smithi045 ceph-mon[102509]: pgmap v527: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:49.576 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:49 smithi188 ceph-mon[112326]: pgmap v527: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:50.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:50.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:50 smithi045 ceph-mon[102509]: from='client.15576 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:50.425 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:50.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:50 smithi188 ceph-mon[112326]: from='client.15576 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:51.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:51 smithi045 ceph-mon[102509]: pgmap v528: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:51.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:51 smithi188 ceph-mon[112326]: pgmap v528: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:51.879 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:51.880 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:52.376 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:53.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:53.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:53 smithi045 ceph-mon[102509]: pgmap v529: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:53.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:53 smithi045 ceph-mon[102509]: from='client.15580 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:53.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:43:53.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:53 smithi188 ceph-mon[112326]: pgmap v529: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:53.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:53 smithi188 ceph-mon[112326]: from='client.15580 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:53.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:43:53.638 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:55.099 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:55.100 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:55.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:55 smithi045 ceph-mon[102509]: pgmap v530: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:55.523 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:55.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:55 smithi188 ceph-mon[112326]: pgmap v530: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:56.524 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:43:56.558 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:56 smithi045 ceph-mon[102509]: from='client.15584 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:56.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:56 smithi188 ceph-mon[112326]: from='client.15584 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:43:56.783 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:43:57.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:57 smithi045 ceph-mon[102509]: pgmap v531: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:57.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:57 smithi188 ceph-mon[112326]: pgmap v531: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:43:58.598 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:43:59.037 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:43:59.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:43:59 smithi045 ceph-mon[102509]: pgmap v532: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:43:59.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:43:59 smithi188 ceph-mon[112326]: pgmap v532: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:00.039 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:00.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:00 smithi045 ceph-mon[102509]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:00.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:00 smithi188 ceph-mon[112326]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:01.384 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:01 smithi045 ceph-mon[102509]: pgmap v533: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:01.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:01 smithi188 ceph-mon[112326]: pgmap v533: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:01.836 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:02.229 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:03.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:03.488 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:03.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:03 smithi188 ceph-mon[112326]: pgmap v534: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:03.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:03 smithi188 ceph-mon[112326]: from='client.15592 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:03.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:03 smithi045 ceph-mon[102509]: pgmap v534: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:03.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:03 smithi045 ceph-mon[102509]: from='client.15592 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:05.024 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:05.024 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:05.309 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:05 smithi045 ceph-mon[102509]: pgmap v535: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:05.475 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:05.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:05 smithi188 ceph-mon[112326]: pgmap v535: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:06.476 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:06.512 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:06 smithi045 ceph-mon[102509]: from='client.15596 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:06.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:06 smithi188 ceph-mon[112326]: from='client.15596 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:07.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:07 smithi045 ceph-mon[102509]: pgmap v536: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:07.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:07 smithi188 ceph-mon[112326]: pgmap v536: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:08.198 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:08.583 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:44:08.603 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:08.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:44:09.604 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:09.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:09 smithi188 ceph-mon[112326]: pgmap v537: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:09.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:09 smithi188 ceph-mon[112326]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:09.638 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:09 smithi045 ceph-mon[102509]: pgmap v537: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:09.639 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:09 smithi045 ceph-mon[102509]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:09.865 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:11.370 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:11.370 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:11.570 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:11 smithi045 ceph-mon[102509]: pgmap v538: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:11.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:11 smithi188 ceph-mon[112326]: pgmap v538: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:11.794 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:12.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:13.052 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:13.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:13 smithi188 ceph-mon[112326]: from='client.15604 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:13.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:13 smithi188 ceph-mon[112326]: pgmap v539: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:13.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:13 smithi045 ceph-mon[102509]: from='client.15604 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:13.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:13 smithi045 ceph-mon[102509]: pgmap v539: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:14.502 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:14.502 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:14.961 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:15.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:15 smithi188 ceph-mon[112326]: pgmap v540: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:15.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:15 smithi045 ceph-mon[102509]: pgmap v540: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:15.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:16.222 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:16.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:16 smithi045 ceph-mon[102509]: from='client.15608 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:16.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:16 smithi188 ceph-mon[112326]: from='client.15608 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:17.307 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:17 smithi045 ceph-mon[102509]: pgmap v541: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:17.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:17 smithi188 ceph-mon[112326]: pgmap v541: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:17.732 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:18.194 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:19.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:19.457 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:19.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:19 smithi188 ceph-mon[112326]: pgmap v542: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:19.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:19 smithi188 ceph-mon[112326]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:19.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:19 smithi045 ceph-mon[102509]: pgmap v542: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:19.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:19 smithi045 ceph-mon[102509]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:20.950 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:20.950 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:21.332 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:21 smithi045 ceph-mon[102509]: pgmap v543: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:21.351 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:21.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:21 smithi188 ceph-mon[112326]: pgmap v543: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:22.352 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:22.610 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:22.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:22 smithi188 ceph-mon[112326]: from='client.15616 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:22.670 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:22 smithi045 ceph-mon[102509]: from='client.15616 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:23.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:23 smithi188 ceph-mon[112326]: pgmap v544: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:23.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:44:23.663 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:23 smithi045 ceph-mon[102509]: pgmap v544: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:23.663 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:44:24.060 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:24.060 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:24.526 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:25.526 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:25.561 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:25 smithi045 ceph-mon[102509]: pgmap v545: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:25.561 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:25 smithi045 ceph-mon[102509]: from='client.15620 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:25.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:25 smithi188 ceph-mon[112326]: pgmap v545: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:25.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:25 smithi188 ceph-mon[112326]: from='client.15620 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:25.786 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:27.254 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:27.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:27 smithi188 ceph-mon[112326]: pgmap v546: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:27.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:27 smithi045 ceph-mon[102509]: pgmap v546: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:27.700 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:28.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:28 smithi188 ceph-mon[112326]: from='client.15624 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:28.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:28 smithi045 ceph-mon[102509]: from='client.15624 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:28.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:28.958 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:29.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:29 smithi188 ceph-mon[112326]: pgmap v547: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:29.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:29 smithi045 ceph-mon[102509]: pgmap v547: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:30.504 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:30.504 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:30.952 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:31.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:31 smithi188 ceph-mon[112326]: pgmap v548: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:31.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:31 smithi045 ceph-mon[102509]: pgmap v548: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:31.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:32.520 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:32 smithi045 ceph-mon[102509]: from='client.15628 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:32.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:32 smithi188 ceph-mon[112326]: from='client.15628 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:33.362 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:33 smithi045 ceph-mon[102509]: pgmap v549: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:33.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:33 smithi188 ceph-mon[112326]: pgmap v549: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:33.761 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:33.761 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:34.191 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:35.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:35.453 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:35.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:35 smithi188 ceph-mon[112326]: pgmap v550: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:35.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:35 smithi188 ceph-mon[112326]: from='client.15632 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:35.670 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:35 smithi045 ceph-mon[102509]: pgmap v550: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:35.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:35 smithi045 ceph-mon[102509]: from='client.15632 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:37.015 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:37.015 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:37.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:37 smithi045 ceph-mon[102509]: pgmap v551: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:37.500 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:37.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:37 smithi188 ceph-mon[112326]: pgmap v551: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:38.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:38.536 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:38 smithi045 ceph-mon[102509]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:38.537 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:44:38.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:38 smithi188 ceph-mon[112326]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:38.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:44:38.761 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:39.608 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:39 smithi045 ceph-mon[102509]: pgmap v552: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:39.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:39 smithi188 ceph-mon[112326]: pgmap v552: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:40.347 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:40.850 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:41.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:41 smithi188 ceph-mon[112326]: pgmap v553: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:41.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:41 smithi045 ceph-mon[102509]: pgmap v553: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:41.851 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:42.108 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:42.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:42 smithi188 ceph-mon[112326]: from='client.15640 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:42.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:42 smithi045 ceph-mon[102509]: from='client.15640 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:43.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:43 smithi188 ceph-mon[112326]: pgmap v554: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:43.655 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:43.656 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:43.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:43 smithi045 ceph-mon[102509]: pgmap v554: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:44.114 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:45.115 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:45.373 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:45 smithi188 ceph-mon[112326]: pgmap v555: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:45 smithi188 ceph-mon[112326]: from='client.15644 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:45.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:45 smithi045 ceph-mon[102509]: pgmap v555: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:45.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:45 smithi045 ceph-mon[102509]: from='client.15644 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:46.484 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:46 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:44:46.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:46 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:44:46.917 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:46.918 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:47.406 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:47.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:47 smithi188 ceph-mon[112326]: pgmap v556: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:47.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:47 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:44:47.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:47 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:44:47.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:47 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:44:47.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:47 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:44:47.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:47 smithi045 ceph-mon[102509]: pgmap v556: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:47.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:47 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:44:47.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:47 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:44:47.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:47 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:44:47.672 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:47 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:44:48.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:48.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:48 smithi188 ceph-mon[112326]: pgmap v557: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:48.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:48 smithi188 ceph-mon[112326]: pgmap v558: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:48.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:48 smithi188 ceph-mon[112326]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:48.664 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:48.670 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:48 smithi045 ceph-mon[102509]: pgmap v557: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:48.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:48 smithi045 ceph-mon[102509]: pgmap v558: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:48.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:48 smithi045 ceph-mon[102509]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:50.156 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:50.157 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:50.608 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:50 smithi045 ceph-mon[102509]: pgmap v559: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:50.628 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:50.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:50 smithi188 ceph-mon[112326]: pgmap v559: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:51.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:51.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:51 smithi188 ceph-mon[112326]: from='client.15652 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:51.665 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:51 smithi045 ceph-mon[102509]: from='client.15652 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:51.888 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:52.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:52 smithi045 ceph-mon[102509]: pgmap v560: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:52.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:52 smithi188 ceph-mon[112326]: pgmap v560: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:53.375 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:53.375 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:53.557 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:53 smithi045 ceph-mon[102509]: pgmap v561: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:53.558 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:44:53.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:53 smithi188 ceph-mon[112326]: pgmap v561: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:53.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:44:53.854 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:54.614 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:54 smithi045 ceph-mon[102509]: from='client.15656 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:54.633 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:54 smithi188 ceph-mon[112326]: from='client.15656 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:54.856 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:55.115 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:55.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:55 smithi188 ceph-mon[112326]: pgmap v562: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:55.671 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:55 smithi045 ceph-mon[102509]: pgmap v562: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:56.548 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:56.548 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:57.011 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:44:58.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:44:58.046 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:57 smithi045 ceph-mon[102509]: pgmap v563: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:58.046 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:57 smithi045 ceph-mon[102509]: from='client.15660 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:58.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:57 smithi188 ceph-mon[112326]: pgmap v563: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:44:58.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:57 smithi188 ceph-mon[112326]: from='client.15660 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:44:58.268 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:44:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:44:59.751 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:44:59.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:44:59 smithi045 ceph-mon[102509]: pgmap v564: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:00.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:44:59 smithi188 ceph-mon[112326]: pgmap v564: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:00.147 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:01.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:00 smithi188 ceph-mon[112326]: from='client.15664 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:01.148 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:01.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:00 smithi045 ceph-mon[102509]: from='client.15664 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:02.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:01 smithi188 ceph-mon[112326]: pgmap v565: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:02.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:01 smithi045 ceph-mon[102509]: pgmap v565: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:02.934 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:02.935 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:03.353 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:04.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:03 smithi188 ceph-mon[112326]: pgmap v566: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:04.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:03 smithi188 ceph-mon[112326]: from='client.15668 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:04.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:03 smithi045 ceph-mon[102509]: pgmap v566: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:04.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:03 smithi045 ceph-mon[102509]: from='client.15668 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:04.355 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:04.613 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:06.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:05 smithi188 ceph-mon[112326]: pgmap v567: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:06.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:05 smithi045 ceph-mon[102509]: pgmap v567: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:06.175 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:06.175 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:06.621 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:06.922 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:06 smithi045 ceph-mon[102509]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:07.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:06 smithi188 ceph-mon[112326]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:07.622 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:07.880 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:07.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:07 smithi045 ceph-mon[102509]: pgmap v568: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:08.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:07 smithi188 ceph-mon[112326]: pgmap v568: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:08.956 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:45:09.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:45:09.382 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:09.383 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:09.812 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:10.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:09 smithi188 ceph-mon[112326]: pgmap v569: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:10.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:09 smithi045 ceph-mon[102509]: pgmap v569: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:10.813 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:11.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:10 smithi188 ceph-mon[112326]: from='client.15676 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:11.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:10 smithi045 ceph-mon[102509]: from='client.15676 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:12.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:11 smithi188 ceph-mon[112326]: pgmap v570: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:12.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:11 smithi045 ceph-mon[102509]: pgmap v570: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:12.602 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:13.001 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:14.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:14.037 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:13 smithi045 ceph-mon[102509]: pgmap v571: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:14.038 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:13 smithi045 ceph-mon[102509]: from='client.15680 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:14.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:13 smithi188 ceph-mon[112326]: pgmap v571: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:14.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:13 smithi188 ceph-mon[112326]: from='client.15680 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:14.261 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:15.781 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:15.781 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:16.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:15 smithi188 ceph-mon[112326]: pgmap v572: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:16.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:15 smithi045 ceph-mon[102509]: pgmap v572: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:16.192 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:17.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:16 smithi188 ceph-mon[112326]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:17.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:16 smithi045 ceph-mon[102509]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:17.193 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:17.451 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:18.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:17 smithi188 ceph-mon[112326]: pgmap v573: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:18.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:17 smithi045 ceph-mon[102509]: pgmap v573: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:18.963 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:18.963 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:19.417 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:20.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:19 smithi188 ceph-mon[112326]: pgmap v574: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:20.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:19 smithi188 ceph-mon[112326]: from='client.15688 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:20.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:19 smithi045 ceph-mon[102509]: pgmap v574: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:20.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:19 smithi045 ceph-mon[102509]: from='client.15688 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:20.420 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:22.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:21 smithi188 ceph-mon[112326]: pgmap v575: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:22.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:21 smithi045 ceph-mon[102509]: pgmap v575: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:22.189 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:22.189 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:22.625 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:22.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:22 smithi045 ceph-mon[102509]: from='client.15692 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:23.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:22 smithi188 ceph-mon[112326]: from='client.15692 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:23.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:23.886 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:23.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:23 smithi045 ceph-mon[102509]: pgmap v576: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:23.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:45:24.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:23 smithi188 ceph-mon[112326]: pgmap v576: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:24.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:45:25.474 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:25.475 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:25.954 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:26.111 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:25 smithi045 ceph-mon[102509]: pgmap v577: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:26.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:25 smithi188 ceph-mon[112326]: pgmap v577: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:26.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:27.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:26 smithi188 ceph-mon[112326]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:27.170 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:26 smithi045 ceph-mon[102509]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:27.213 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:28.030 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:27 smithi045 ceph-mon[102509]: pgmap v578: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:28.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:27 smithi188 ceph-mon[112326]: pgmap v578: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:28.712 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:28.712 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:29.177 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:30.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:29 smithi188 ceph-mon[112326]: pgmap v579: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:30.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:29 smithi188 ceph-mon[112326]: from='client.15700 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:30.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:29 smithi045 ceph-mon[102509]: pgmap v579: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:30.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:29 smithi045 ceph-mon[102509]: from='client.15700 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:30.178 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:30.431 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:31.827 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:32.026 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:31 smithi045 ceph-mon[102509]: pgmap v580: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:32.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:31 smithi188 ceph-mon[112326]: pgmap v580: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:32.240 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:33.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:32 smithi188 ceph-mon[112326]: from='client.15704 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:33.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:32 smithi045 ceph-mon[102509]: from='client.15704 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:33.241 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:33.498 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:34.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:33 smithi188 ceph-mon[112326]: pgmap v581: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:34.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:33 smithi045 ceph-mon[102509]: pgmap v581: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:35.043 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:35.043 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:35.556 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:35.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:35 smithi188 ceph-mon[112326]: pgmap v582: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:35.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:35 smithi188 ceph-mon[112326]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:35.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:35 smithi045 ceph-mon[102509]: pgmap v582: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:35.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:35 smithi045 ceph-mon[102509]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:36.557 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:36.814 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:37.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:37 smithi045 ceph-mon[102509]: pgmap v583: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:38.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:37 smithi188 ceph-mon[112326]: pgmap v583: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:38.329 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:38.329 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:38.807 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:39.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:45:39.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:45:39.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:40.069 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:40.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:39 smithi188 ceph-mon[112326]: from='client.15712 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:40.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:39 smithi188 ceph-mon[112326]: pgmap v584: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:40.170 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:39 smithi045 ceph-mon[102509]: from='client.15712 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:40.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:39 smithi045 ceph-mon[102509]: pgmap v584: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:41.657 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:41.658 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:42.061 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:41 smithi045 ceph-mon[102509]: pgmap v585: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:42.080 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:42.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:41 smithi188 ceph-mon[112326]: pgmap v585: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:43.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:43.116 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:42 smithi045 ceph-mon[102509]: from='client.15716 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:43.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:42 smithi188 ceph-mon[112326]: from='client.15716 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:43.921 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:43 smithi045 ceph-mon[102509]: pgmap v586: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:44.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:43 smithi188 ceph-mon[112326]: pgmap v586: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:44.940 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:44.941 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:45.441 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:46.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:45 smithi188 ceph-mon[112326]: pgmap v587: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:46.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:45 smithi188 ceph-mon[112326]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:46.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:45 smithi045 ceph-mon[102509]: pgmap v587: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:46.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:45 smithi045 ceph-mon[102509]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:46.443 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:47.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:46 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:45:47.170 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:46 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:45:48.091 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:47 smithi045 ceph-mon[102509]: pgmap v588: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:48.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:47 smithi188 ceph-mon[112326]: pgmap v588: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:48.501 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:48.501 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:35:36.673118Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:35:36.672896Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:35:36.672979Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:35:36.673183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:35:36.673572Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:35:36.673613Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:35:36.672822Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:35:36.672667Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:35:36.673051Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:35:36.673313Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:35:36.673248Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:49.054 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:50.055 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:50.083 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:49 smithi045 ceph-mon[102509]: pgmap v589: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:50.084 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:49 smithi045 ceph-mon[102509]: from='client.15724 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:50.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:49 smithi188 ceph-mon[112326]: pgmap v589: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:50.133 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:49 smithi188 ceph-mon[112326]: from='client.15724 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:51.940 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:51.940 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:52.318 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:52 smithi045 ceph-mon[102509]: pgmap v590: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:52.318 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:45:52.318 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:45:52.319 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:45:52.319 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:45:52.319 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:45:52.319 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:45:52.338 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:52.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:52 smithi188 ceph-mon[112326]: pgmap v590: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:45:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:45:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:45:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:45:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:45:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:45:53.339 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:53.373 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:53 smithi045 ceph-mon[102509]: pgmap v591: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:53.374 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:53 smithi045 ceph-mon[102509]: pgmap v592: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:53.374 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:53 smithi045 ceph-mon[102509]: from='client.15728 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:53.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:53 smithi188 ceph-mon[112326]: pgmap v591: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:53.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:53 smithi188 ceph-mon[112326]: pgmap v592: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:53.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:53 smithi188 ceph-mon[112326]: from='client.15728 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:54.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:54 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:45:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:54 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:45:55.150 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:55.150 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:55.299 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:55 smithi045 ceph-mon[102509]: pgmap v593: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:55.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:55 smithi188 ceph-mon[112326]: pgmap v593: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:55.562 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:56.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:56 smithi188 ceph-mon[112326]: from='client.15732 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:56.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:56 smithi045 ceph-mon[102509]: from='client.15732 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:45:56.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:45:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:45:57.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:57 smithi188 ceph-mon[112326]: pgmap v594: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:57.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:57 smithi045 ceph-mon[102509]: pgmap v594: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:58.323 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:45:58.324 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:45:58.745 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:45:59.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:45:59 smithi188 ceph-mon[112326]: pgmap v595: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:59.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:45:59 smithi045 ceph-mon[102509]: pgmap v595: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:45:59.746 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:00.004 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:00.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:00 smithi045 ceph-mon[102509]: from='client.15736 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:00.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:00 smithi188 ceph-mon[112326]: from='client.15736 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:01.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:01 smithi188 ceph-mon[112326]: pgmap v596: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:01.141 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:01 smithi045 ceph-mon[102509]: pgmap v596: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:01.566 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:01.566 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:02.004 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:03.005 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:03.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:03 smithi188 ceph-mon[112326]: pgmap v597: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:03.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:03 smithi188 ceph-mon[112326]: from='client.15740 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:03.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:03 smithi045 ceph-mon[102509]: pgmap v597: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:03.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:03 smithi045 ceph-mon[102509]: from='client.15740 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:04.842 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:04.843 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:05.170 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:05 smithi045 ceph-mon[102509]: pgmap v598: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:05.333 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:05.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:05 smithi188 ceph-mon[112326]: pgmap v598: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:06.132 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:06 smithi188 ceph-mon[112326]: from='client.15744 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:06.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:06.370 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:06 smithi045 ceph-mon[102509]: from='client.15744 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:07.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:07 smithi045 ceph-mon[102509]: pgmap v599: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:07.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:07 smithi188 ceph-mon[112326]: pgmap v599: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:08.099 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:08.099 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:08.535 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:09.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:09 smithi188 ceph-mon[112326]: pgmap v600: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:09.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:09 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:46:09.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:09 smithi188 ceph-mon[112326]: from='client.15748 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:09.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:09 smithi045 ceph-mon[102509]: pgmap v600: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:09.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:09 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:46:09.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:09 smithi045 ceph-mon[102509]: from='client.15748 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:09.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:11.080 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:11 smithi045 ceph-mon[102509]: pgmap v601: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:11.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:11 smithi188 ceph-mon[112326]: pgmap v601: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:11.497 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:11.497 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:11.999 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:13.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:13.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:13 smithi188 ceph-mon[112326]: pgmap v602: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:13.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:13 smithi188 ceph-mon[112326]: from='client.15752 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:13.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:13 smithi045 ceph-mon[102509]: pgmap v602: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:13.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:13 smithi045 ceph-mon[102509]: from='client.15752 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:14.728 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:14.729 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:15.216 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:15.310 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:15 smithi045 ceph-mon[102509]: pgmap v603: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:15.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:15 smithi188 ceph-mon[112326]: pgmap v603: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:16.217 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:16.384 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:16 smithi188 ceph-mon[112326]: from='client.15756 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:16.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:16 smithi045 ceph-mon[102509]: from='client.15756 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:16.474 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:17.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:17 smithi188 ceph-mon[112326]: pgmap v604: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:17.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:17 smithi045 ceph-mon[102509]: pgmap v604: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:17.954 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:17.954 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:18.411 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:19.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:19 smithi188 ceph-mon[112326]: pgmap v605: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:19.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:19 smithi188 ceph-mon[112326]: from='client.15760 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:19.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:19.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:19 smithi045 ceph-mon[102509]: pgmap v605: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:19.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:19 smithi045 ceph-mon[102509]: from='client.15760 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:21.157 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:21.158 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:21.343 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:21 smithi045 ceph-mon[102509]: pgmap v606: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:21.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:21 smithi188 ceph-mon[112326]: pgmap v606: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:21.606 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:22.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:22 smithi188 ceph-mon[112326]: from='client.15764 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:22.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:22 smithi045 ceph-mon[102509]: from='client.15764 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:22.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:22.865 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:23.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:23 smithi188 ceph-mon[112326]: pgmap v607: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:23.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:23 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:46:23.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:23 smithi045 ceph-mon[102509]: pgmap v607: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:23.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:23 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:46:24.331 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:24.332 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:24.825 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:25.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:25 smithi188 ceph-mon[112326]: pgmap v608: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:25.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:25 smithi045 ceph-mon[102509]: pgmap v608: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:25.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:26.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:26 smithi045 ceph-mon[102509]: from='client.15768 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:26.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:26 smithi188 ceph-mon[112326]: from='client.15768 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:27.119 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:27 smithi045 ceph-mon[102509]: pgmap v609: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:27.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:27 smithi188 ceph-mon[112326]: pgmap v609: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:27.559 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:28.035 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:29.036 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:29.295 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:29.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:29 smithi188 ceph-mon[112326]: pgmap v610: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:29.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:29 smithi188 ceph-mon[112326]: from='client.15772 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:29.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:29 smithi045 ceph-mon[102509]: pgmap v610: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:29.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:29 smithi045 ceph-mon[102509]: from='client.15772 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:30.772 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:30.772 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:31.232 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:31.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:31 smithi188 ceph-mon[112326]: pgmap v611: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:31.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:31 smithi045 ceph-mon[102509]: pgmap v611: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:32.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:32.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:32 smithi188 ceph-mon[112326]: from='client.15776 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:32.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:32 smithi045 ceph-mon[102509]: from='client.15776 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:32.492 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:33.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:33 smithi188 ceph-mon[112326]: pgmap v612: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:33.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:33 smithi045 ceph-mon[102509]: pgmap v612: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:33.965 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:33.965 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:34.427 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:35.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:35 smithi188 ceph-mon[112326]: pgmap v613: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:35.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:35 smithi188 ceph-mon[112326]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:35.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:35 smithi045 ceph-mon[102509]: pgmap v613: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:35.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:35 smithi045 ceph-mon[102509]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:35.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:35.688 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:37.139 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:37.140 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:37.349 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:37 smithi045 ceph-mon[102509]: pgmap v614: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:37.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:37 smithi188 ceph-mon[112326]: pgmap v614: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:37.618 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:38.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:38 smithi188 ceph-mon[112326]: from='client.15784 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:38.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:38 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:46:38.424 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:38 smithi045 ceph-mon[102509]: from='client.15784 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:38.424 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:38 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:46:38.620 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:38.877 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:39.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:39 smithi188 ceph-mon[112326]: pgmap v615: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:39.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:39 smithi045 ceph-mon[102509]: pgmap v615: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:40.279 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:40.704 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:41.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:41 smithi188 ceph-mon[112326]: pgmap v616: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:41.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:41 smithi188 ceph-mon[112326]: from='client.15788 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:41.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:41 smithi045 ceph-mon[102509]: pgmap v616: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:41.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:41 smithi045 ceph-mon[102509]: from='client.15788 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:41.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:41.966 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:43.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:43 smithi188 ceph-mon[112326]: pgmap v617: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:43.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:43 smithi045 ceph-mon[102509]: pgmap v617: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:43.532 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:43.533 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:44.008 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:45.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:45.269 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:45.310 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:45 smithi045 ceph-mon[102509]: pgmap v618: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:45.310 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:45 smithi045 ceph-mon[102509]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:45.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:45 smithi188 ceph-mon[112326]: pgmap v618: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:45.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:45 smithi188 ceph-mon[112326]: from='client.15792 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:46.787 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:46.788 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:47.279 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:47.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:47 smithi188 ceph-mon[112326]: pgmap v619: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:47.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:47 smithi045 ceph-mon[102509]: pgmap v619: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:48.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:48.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:48 smithi188 ceph-mon[112326]: from='client.15796 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:48.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:48 smithi045 ceph-mon[102509]: from='client.15796 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:48.540 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:49.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:49 smithi188 ceph-mon[112326]: pgmap v620: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:49.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:49 smithi045 ceph-mon[102509]: pgmap v620: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:49.937 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:49.937 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:50.406 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:51.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:51 smithi188 ceph-mon[112326]: pgmap v621: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:51.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:51 smithi188 ceph-mon[112326]: from='client.15800 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:51.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:51.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:51 smithi045 ceph-mon[102509]: pgmap v621: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:51.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:51 smithi045 ceph-mon[102509]: from='client.15800 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:52.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:46:52.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:46:52.171 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:52 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:46:52.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-24T09:46:52.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:46:52.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:52 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:46:53.126 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:53.126 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:53.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:53 smithi188 ceph-mon[112326]: pgmap v622: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:53.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:46:53.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:46:53.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:46:53.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:46:53.383 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:53 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:46:53.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:53 smithi045 ceph-mon[102509]: pgmap v622: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:53.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-24T09:46:53.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-24T09:46:53.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' 2023-06-24T09:46:53.422 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-24T09:46:53.423 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:53 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:46:53.663 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:54 smithi188 ceph-mon[112326]: pgmap v623: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:54 smithi188 ceph-mon[112326]: pgmap v624: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:54.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:54 smithi188 ceph-mon[112326]: from='client.15804 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:54.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:54 smithi045 ceph-mon[102509]: pgmap v623: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:54.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:54 smithi045 ceph-mon[102509]: pgmap v624: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:54.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:54 smithi045 ceph-mon[102509]: from='client.15804 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:54.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:54.920 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:56.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:56 smithi188 ceph-mon[112326]: pgmap v625: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:56.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:56 smithi045 ceph-mon[102509]: pgmap v625: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:56.476 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:56.476 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:46:56.969 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:46:57.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:46:58.227 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:46:58.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:58 smithi188 ceph-mon[112326]: from='client.15808 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:58.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:46:58 smithi188 ceph-mon[112326]: pgmap v626: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:58.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:58 smithi045 ceph-mon[102509]: from='client.15808 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:46:58.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:46:58 smithi045 ceph-mon[102509]: pgmap v626: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:46:59.776 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:46:59.777 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:47:00.230 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:47:00.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:00 smithi045 ceph-mon[102509]: pgmap v627: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:00.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:00 smithi188 ceph-mon[112326]: pgmap v627: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:01.231 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:47:01.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:01 smithi045 ceph-mon[102509]: from='client.15812 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:47:01.493 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:47:01.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:01 smithi188 ceph-mon[112326]: from='client.15812 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:47:02.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:02 smithi045 ceph-mon[102509]: pgmap v628: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:02.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:02 smithi188 ceph-mon[112326]: pgmap v628: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:02.961 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:47:02.961 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:47:03.360 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:47:04.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:47:04.397 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:04 smithi045 ceph-mon[102509]: pgmap v629: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:04.397 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:04 smithi045 ceph-mon[102509]: from='client.15816 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:47:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:47:04.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:04 smithi188 ceph-mon[112326]: pgmap v629: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:04.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:04 smithi188 ceph-mon[112326]: from='client.15816 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:47:06.155 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:47:06.155 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:47:06.409 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:06 smithi188 ceph-mon[112326]: pgmap v630: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:06.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:06 smithi045 ceph-mon[102509]: pgmap v630: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:06.576 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:47:07.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:07 smithi045 ceph-mon[102509]: from='client.15820 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:47:07.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:47:07.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:07 smithi188 ceph-mon[112326]: from='client.15820 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:47:07.836 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:47:08.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:08 smithi045 ceph-mon[102509]: pgmap v631: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:08.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:08 smithi045 ceph-mon[102509]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:47:08.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:08 smithi188 ceph-mon[112326]: pgmap v631: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:08.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:08 smithi188 ceph-mon[112326]: from='mgr.14229 172.21.15.45:0/2431077895' entity='mgr.smithi045.dcgntd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2023-06-24T09:47:09.287 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:47:09.288 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:47:09.744 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:47:10.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:10 smithi045 ceph-mon[102509]: pgmap v632: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:10.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:10 smithi045 ceph-mon[102509]: from='client.15824 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:47:10.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:10 smithi188 ceph-mon[112326]: pgmap v632: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:10.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:10 smithi188 ceph-mon[112326]: from='client.15824 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:47:10.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:47:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:47:12.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:12 smithi045 ceph-mon[102509]: pgmap v633: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:12.492 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:47:12.493 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:47:12.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:12 smithi188 ceph-mon[112326]: pgmap v633: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:12.918 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:47:13.919 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fceaf654-1270-11ee-9b2b-001a4aab830c -- ceph orch ls -f json 2023-06-24T09:47:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/mon.smithi045/config 2023-06-24T09:47:14.420 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:14 smithi045 ceph-mon[102509]: from='client.15828 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:47:14.421 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:14 smithi045 ceph-mon[102509]: pgmap v634: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:14.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:14 smithi188 ceph-mon[112326]: from='client.15828 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-24T09:47:14.632 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:14 smithi188 ceph-mon[112326]: pgmap v634: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:15.652 INFO:teuthology.orchestra.run.smithi045.stdout: 2023-06-24T09:47:15.652 INFO:teuthology.orchestra.run.smithi045.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2023-06-24T09:26:19.999744Z", "last_refresh": "2023-06-24T09:45:50.997870Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:01.373137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2023-06-24T09:26:15.322444Z", "last_refresh": "2023-06-24T09:37:41.526560Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:03.451725Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2023-06-24T09:26:14.333167Z", "last_refresh": "2023-06-24T09:37:41.526814Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:30:36.249379Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2023-06-24T09:30:36.245974Z", "last_refresh": "2023-06-24T09:37:41.527595Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2023-06-24T09:26:17.823833Z", "last_refresh": "2023-06-24T09:45:50.997934Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.257281Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2023-06-24T09:30:36.254810Z", "last_refresh": "2023-06-24T09:37:41.527540Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2023-06-24T09:30:36.251989Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2023-06-24T09:30:36.249639Z", "last_refresh": "2023-06-24T09:45:50.998379Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:30:36.254598Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2023-06-24T09:30:36.252206Z", "last_refresh": "2023-06-24T09:37:41.527679Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2023-06-24T09:28:09.612000Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2023-06-24T09:26:13.231635Z", "last_refresh": "2023-06-24T09:37:41.527015Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:17.908328Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi045:172.21.15.45=smithi045", "smithi188:172.21.15.188=smithi188"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2023-06-24T09:26:58.570921Z", "last_refresh": "2023-06-24T09:37:41.527105Z", "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:06.980205Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2023-06-24T09:26:18.995113Z", "last_refresh": "2023-06-24T09:37:41.526908Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2023-06-24T09:28:23.735219Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2023-06-24T09:28:23.731591Z", "last_refresh": "2023-06-24T09:37:41.527187Z", "running": 8, "size": 8}}, {"events": ["2023-06-24T09:28:17.918147Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2023-06-24T09:26:16.810059Z", "last_refresh": "2023-06-24T09:45:50.998006Z", "ports": [9095], "running": 1, "size": 1}}] 2023-06-24T09:47:16.108 INFO:tasks.cephadm:jaeger-agent has 0/2 2023-06-24T09:47:16.108 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 84, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_b9d28de6006117997208584eb86d8d304f2a113d/qa/tasks/cephadm.py", line 1291, in wait_for_service while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (301) after waiting for 300 seconds 2023-06-24T09:47:16.181 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b1b3d37e526e40ef8ee020870c64f7ea Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 84, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_b9d28de6006117997208584eb86d8d304f2a113d/qa/tasks/cephadm.py", line 1291, in wait_for_service while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (301) after waiting for 300 seconds 2023-06-24T09:47:16.184 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-06-24T09:47:16.193 INFO:tasks.cephadm:Teardown begin 2023-06-24T09:47:16.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-24T09:47:16.225 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:16 smithi045 ceph-mon[102509]: pgmap v635: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:16.229 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-24T09:47:16.268 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-06-24T09:47:16.269 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-06-24T09:47:16.289 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-06-24T09:47:16.330 INFO:tasks.cephadm:Stopping all daemons... 2023-06-24T09:47:16.330 INFO:tasks.cephadm.mon.smithi045:Stopping mon.smithi045... 2023-06-24T09:47:16.331 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mon.smithi045 2023-06-24T09:47:16.503 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:16 smithi188 ceph-mon[112326]: pgmap v635: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-24T09:47:16.559 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:16 smithi045 systemd[1]: Stopping Ceph mon.smithi045 for fceaf654-1270-11ee-9b2b-001a4aab830c... 2023-06-24T09:47:16.860 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:16 smithi045 ceph-fceaf654-1270-11ee-9b2b-001a4aab830c-mon-smithi045[102505]: 2023-06-24T09:47:16.557+0000 7f7dc2c49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi045 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-24T09:47:16.861 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:16 smithi045 ceph-fceaf654-1270-11ee-9b2b-001a4aab830c-mon-smithi045[102505]: 2023-06-24T09:47:16.557+0000 7f7dc2c49700 -1 mon.smithi045@0(leader) e2 *** Got Signal Terminated *** 2023-06-24T09:47:17.172 INFO:journalctl@ceph.mon.smithi045.smithi045.stdout:Jun 24 09:47:16 smithi045 bash[208682]: ceph-fceaf654-1270-11ee-9b2b-001a4aab830c-mon-smithi045 2023-06-24T09:47:17.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mon.smithi045.service' 2023-06-24T09:47:17.536 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-24T09:47:17.537 INFO:tasks.cephadm.mon.smithi045:Stopped mon.smithi045 2023-06-24T09:47:17.537 INFO:tasks.cephadm.mon.smithi188:Stopping mon.smithi188... 2023-06-24T09:47:17.537 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mon.smithi188 2023-06-24T09:47:17.882 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:17 smithi188 systemd[1]: Stopping Ceph mon.smithi188 for fceaf654-1270-11ee-9b2b-001a4aab830c... 2023-06-24T09:47:17.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:17 smithi188 ceph-fceaf654-1270-11ee-9b2b-001a4aab830c-mon-smithi188[112320]: 2023-06-24T09:47:17.767+0000 7fe8a9292700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi188 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-24T09:47:17.883 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:17 smithi188 ceph-fceaf654-1270-11ee-9b2b-001a4aab830c-mon-smithi188[112320]: 2023-06-24T09:47:17.767+0000 7fe8a9292700 -1 mon.smithi188@1(peon) e2 *** Got Signal Terminated *** 2023-06-24T09:47:18.382 INFO:journalctl@ceph.mon.smithi188.smithi188.stdout:Jun 24 09:47:18 smithi188 bash[136918]: ceph-fceaf654-1270-11ee-9b2b-001a4aab830c-mon-smithi188 2023-06-24T09:47:18.519 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fceaf654-1270-11ee-9b2b-001a4aab830c@mon.smithi188.service' 2023-06-24T09:47:18.582 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-24T09:47:18.582 INFO:tasks.cephadm.mon.smithi188:Stopped mon.smithi188 2023-06-24T09:47:18.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fceaf654-1270-11ee-9b2b-001a4aab830c --force --keep-logs 2023-06-24T09:47:57.798 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fceaf654-1270-11ee-9b2b-001a4aab830c --force --keep-logs 2023-06-24T09:48:33.253 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-24T09:48:33.293 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-24T09:48:33.324 INFO:tasks.cephadm:Archiving crash dumps... 2023-06-24T09:48:33.326 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/crash to /home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893/remote/smithi045/crash 2023-06-24T09:48:33.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/crash -- . 2023-06-24T09:48:33.386 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/crash: Cannot open: No such file or directory 2023-06-24T09:48:33.386 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2023-06-24T09:48:33.388 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/crash to /home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893/remote/smithi188/crash 2023-06-24T09:48:33.389 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/crash -- . 2023-06-24T09:48:33.421 INFO:teuthology.orchestra.run.smithi188.stderr:tar: /var/lib/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/crash: Cannot open: No such file or directory 2023-06-24T09:48:33.422 INFO:teuthology.orchestra.run.smithi188.stderr:tar: Error is not recoverable: exiting now 2023-06-24T09:48:33.423 INFO:tasks.cephadm:Checking cluster log for badness... 2023-06-24T09:48:33.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2023-06-24T09:48:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:grep: /var/log/ceph/fceaf654-1270-11ee-9b2b-001a4aab830c/ceph.log: No such file or directory 2023-06-24T09:48:33.456 INFO:tasks.cephadm:Compressing logs... 2023-06-24T09:48:33.457 DEBUG:teuthology.orchestra.run.smithi045:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-24T09:48:33.499 DEBUG:teuthology.orchestra.run.smithi188:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-24T09:48:33.529 INFO:teuthology.orchestra.run.smithi188.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-06-24T09:48:33.529 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-06-24T09:48:35.212 INFO:tasks.cephadm:Archiving logs... 2023-06-24T09:48:35.212 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893/remote/smithi045/log 2023-06-24T09:48:35.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-24T09:48:35.483 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893/remote/smithi188/log 2023-06-24T09:48:35.484 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-24T09:48:35.616 INFO:tasks.cephadm:Removing cluster... 2023-06-24T09:48:35.617 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fceaf654-1270-11ee-9b2b-001a4aab830c --force 2023-06-24T09:48:35.961 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fceaf654-1270-11ee-9b2b-001a4aab830c --force 2023-06-24T09:48:36.308 INFO:tasks.cephadm:Removing cephadm ... 2023-06-24T09:48:36.308 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2023-06-24T09:48:36.328 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf /home/ubuntu/cephtest/cephadm 2023-06-24T09:48:36.347 INFO:tasks.cephadm:Teardown complete 2023-06-24T09:48:36.347 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2023-06-24T09:48:36.359 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi045:/dev/vg_nvme/lv_1... 2023-06-24T09:48:36.359 DEBUG:teuthology.orchestra.run.smithi045:> sudo nvme disconnect -n lv_1 2023-06-24T09:48:36.504 INFO:teuthology.orchestra.run.smithi045.stdout:NQN:lv_1 disconnected 1 controller(s) 2023-06-24T09:48:36.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:36.506 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi045:/dev/vg_nvme/lv_2... 2023-06-24T09:48:36.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo nvme disconnect -n lv_2 2023-06-24T09:48:36.630 INFO:teuthology.orchestra.run.smithi045.stdout:NQN:lv_2 disconnected 1 controller(s) 2023-06-24T09:48:36.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:36.631 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi045:/dev/vg_nvme/lv_3... 2023-06-24T09:48:36.632 DEBUG:teuthology.orchestra.run.smithi045:> sudo nvme disconnect -n lv_3 2023-06-24T09:48:36.754 INFO:teuthology.orchestra.run.smithi045.stdout:NQN:lv_3 disconnected 1 controller(s) 2023-06-24T09:48:36.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:36.755 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi045:/dev/vg_nvme/lv_4... 2023-06-24T09:48:36.756 DEBUG:teuthology.orchestra.run.smithi045:> sudo nvme disconnect -n lv_4 2023-06-24T09:48:36.877 INFO:teuthology.orchestra.run.smithi045.stdout:NQN:lv_4 disconnected 1 controller(s) 2023-06-24T09:48:36.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:36.881 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2023-06-24T09:48:36.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/scratch_devs 2023-06-24T09:48:36.911 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi188:/dev/vg_nvme/lv_1... 2023-06-24T09:48:36.912 DEBUG:teuthology.orchestra.run.smithi188:> sudo nvme disconnect -n lv_1 2023-06-24T09:48:37.035 INFO:teuthology.orchestra.run.smithi188.stdout:NQN:lv_1 disconnected 1 controller(s) 2023-06-24T09:48:37.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:37.038 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi188:/dev/vg_nvme/lv_2... 2023-06-24T09:48:37.038 DEBUG:teuthology.orchestra.run.smithi188:> sudo nvme disconnect -n lv_2 2023-06-24T09:48:37.158 INFO:teuthology.orchestra.run.smithi188.stdout:NQN:lv_2 disconnected 1 controller(s) 2023-06-24T09:48:37.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:37.160 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi188:/dev/vg_nvme/lv_3... 2023-06-24T09:48:37.160 DEBUG:teuthology.orchestra.run.smithi188:> sudo nvme disconnect -n lv_3 2023-06-24T09:48:37.298 INFO:teuthology.orchestra.run.smithi188.stdout:NQN:lv_3 disconnected 1 controller(s) 2023-06-24T09:48:37.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:37.299 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi188:/dev/vg_nvme/lv_4... 2023-06-24T09:48:37.300 DEBUG:teuthology.orchestra.run.smithi188:> sudo nvme disconnect -n lv_4 2023-06-24T09:48:37.441 INFO:teuthology.orchestra.run.smithi188.stdout:NQN:lv_4 disconnected 1 controller(s) 2023-06-24T09:48:37.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:37.442 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-06-24T09:48:37.443 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/scratch_devs 2023-06-24T09:48:37.469 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-06-24T09:48:37.479 INFO:teuthology.task.clock:Checking final clock skew... 2023-06-24T09:48:37.480 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-24T09:48:37.483 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-24T09:48:37.501 INFO:teuthology.orchestra.run.smithi045.stderr:bash: ntpq: command not found 2023-06-24T09:48:37.505 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-24T09:48:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2023-06-24T09:48:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 9 +3861us[+3874us] +/- 87ms 2023-06-24T09:48:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv02.front.sepia.ceph.com 3 6 377 7 -1910us[-1896us] +/- 61ms 2023-06-24T09:48:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 -789us[ -775us] +/- 65ms 2023-06-24T09:48:37.506 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-06-24T09:48:37.526 INFO:teuthology.orchestra.run.smithi188.stderr:bash: ntpq: command not found 2023-06-24T09:48:37.530 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-24T09:48:37.531 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2023-06-24T09:48:37.531 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 4 +3887us[+3887us] +/- 87ms 2023-06-24T09:48:37.531 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv02.front.sepia.ceph.com 3 6 377 7 -1886us[-1897us] +/- 61ms 2023-06-24T09:48:37.531 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -777us[ -788us] +/- 65ms 2023-06-24T09:48:37.532 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-06-24T09:48:37.533 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-06-24T09:48:37.543 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-06-24T09:48:37.544 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-06-24T09:48:37.555 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-24T09:48:37.664 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-24T09:48:37.753 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-24T09:48:37.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:37.789 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-24T09:48:37.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:37.821 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-06-24T09:48:37.831 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-06-24T09:48:37.840 INFO:teuthology.task.internal:Duration was 1860.386533 seconds 2023-06-24T09:48:37.840 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-06-24T09:48:37.862 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-06-24T09:48:37.862 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-24T09:48:37.865 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-24T09:48:37.912 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-24T09:48:37.913 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-24T09:48:38.182 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-06-24T09:48:38.183 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2023-06-24T09:48:38.183 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-24T09:48:38.208 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2023-06-24T09:48:38.209 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-24T09:48:38.232 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-06-24T09:48:38.233 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-24T09:48:38.250 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-24T09:48:38.319 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-06-24T09:48:38.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-24T09:48:38.322 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-24T09:48:38.488 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-06-24T09:48:38.500 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-06-24T09:48:38.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-24T09:48:38.529 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-24T09:48:38.557 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-06-24T09:48:38.568 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-24T09:48:38.571 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-24T09:48:38.596 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2023-06-24T09:48:38.623 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2023-06-24T09:48:38.664 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-24T09:48:38.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:38.687 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-24T09:48:38.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-24T09:48:38.723 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-06-24T09:48:38.733 INFO:teuthology.task.internal:Transferring archived files... 2023-06-24T09:48:38.735 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893/remote/smithi045 2023-06-24T09:48:38.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-24T09:48:38.786 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893/remote/smithi188 2023-06-24T09:48:38.786 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-24T09:48:38.825 INFO:teuthology.task.internal:Removing archive directory... 2023-06-24T09:48:38.826 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-24T09:48:38.828 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-24T09:48:38.877 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-06-24T09:48:38.887 INFO:teuthology.task.internal:Not uploading archives. 2023-06-24T09:48:38.888 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-06-24T09:48:38.898 INFO:teuthology.task.internal:Tidying up after the test... 2023-06-24T09:48:38.899 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-24T09:48:38.901 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-24T09:48:38.917 INFO:teuthology.orchestra.run.smithi045.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 24 09:48 /home/ubuntu/cephtest 2023-06-24T09:48:38.934 INFO:teuthology.orchestra.run.smithi188.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 24 09:48 /home/ubuntu/cephtest 2023-06-24T09:48:38.935 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-06-24T09:48:38.946 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-06-24T09:48:38.973 INFO:teuthology.nuke:Checking targets against current locks 2023-06-24T09:48:39.003 DEBUG:teuthology.nuke:shortname: smithi045 2023-06-24T09:48:39.004 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-24T09:48:39.028 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-24 09:09:08.263027', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNBaO0t3/+SbaU6HMFkYX0kuNEYQm4jLZRjU6sCF0TOfNmkjFCbVqgn9nX1TpSuMGZ5vO2qqDLiNWNSOhhvv97w='} 2023-06-24T09:48:39.031 DEBUG:teuthology.nuke:shortname: smithi188 2023-06-24T09:48:39.031 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-24T09:48:39.051 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-23_20:59:31-orch-reef-distro-default-smithi/7314893', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-24 09:09:08.260994', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD7mS1xfBOZDSTrkogxzQbG45xjsup+Y1b0h1In6iQjviqgcryJ+dyuLLMSAEMEPadkax4Q0EsbG2EVAtWdCVqZTxSodnboVwNrg4fg1tdiMtFLjFwXBoC3Q8iGNL4bGZsIbGyPyWBbTRkWNb6AKwF6YNvmQ7CramB1Wl4QSG6XztCAgnx1zD+2/afSzyLUNM9PK0F+ZiOxYRazSEkdj4YeSC0mWZ/2JbmFNNBhlt2tc/m+1odteQQZPz+R1oJ8Cz7+uYWilaeKFUYmxjZ98luem009g3HK0ffALugDsywZ+1Q1PPHWVF1ieSthmQaXUHe/JqBneGPJa7Wf7iWA7eFcx9e+wcv4jPHXZXY1n8FlqcO6uNXx+ivKvZubj31hmNGDPMiJoy4c5Vj2taNp+VCiShFia7lqElC8uULdXvTLAtg/lBxzwoN1CRHdTtrFGERs5W+nd6MyqWnxV4srLi2encB1DSqedtSYX/ON2vHBzaMN1H5DZqqYwklSlavwakk='} 2023-06-24T09:48:39.087 INFO:teuthology.orchestra.console:Power off smithi045 2023-06-24T09:48:39.088 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-24T09:48:39.108 INFO:teuthology.orchestra.console:Power off smithi188 2023-06-24T09:48:39.109 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-24T09:48:39.124 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-24T09:48:39.135 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-24T09:48:43.148 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-24T09:48:43.161 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-24T09:48:47.264 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-24T09:48:47.274 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-24T09:48:47.478 INFO:teuthology.orchestra.console:Power off for smithi045 completed 2023-06-24T09:48:47.489 INFO:teuthology.orchestra.console:Power off for smithi188 completed 2023-06-24T09:48:47.594 INFO:teuthology.run:Summary data: description: orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/jaeger 3-final} duration: 1860.3865327835083 failure_reason: reached maximum tries (301) after waiting for 300 seconds owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b1b3d37e526e40ef8ee020870c64f7ea status: fail success: false 2023-06-24T09:48:47.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-24T09:48:47.685 INFO:teuthology.run:FAIL