2024-03-12T01:58:41.969 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-12T01:58:41.969 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248 --owner scheduled_teuthology@teuthology --name teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi --description orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/jaeger 3-final} -- /home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248/orig.config.yaml 2024-03-12T01:58:41.996 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T01:58:42.078 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248 branch: reef description: orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/jaeger 3-final} email: ceph-qa@ceph.com first_in_suite: false job_id: '7592248' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: reef ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: c2ea977b6956d1f694ccf66acc692fdd979e3baa ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: c2ea977b6956d1f694ccf66acc692fdd979e3baa kernel: hwe: true workunit: branch: reef sha1: 583c7cd94fcfcfa9abcbbb46bafd029ba8504cb0 owner: scheduled_teuthology@teuthology priority: 180 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - - host.b - client.1 seed: 3254 sha1: c2ea977b6956d1f694ccf66acc692fdd979e3baa sleep_before_teardown: 0 subset: 44/64 suite: orch suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_583c7cd94fcfcfa9abcbbb46bafd029ba8504cb0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 583c7cd94fcfcfa9abcbbb46bafd029ba8504cb0 targets: smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - nvme_loop: null - cephadm: roleless: true - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph orch apply jaeger - cephadm.wait_for_service: service: elasticsearch - cephadm.wait_for_service: service: jaeger-collector - cephadm.wait_for_service: service: jaeger-query - cephadm.wait_for_service: service: jaeger-agent - cephadm.shell: host.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls | grep '^osd.all-available-devices ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-11_22:08:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-12T01:58:42.079 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_583c7cd94fcfcfa9abcbbb46bafd029ba8504cb0/qa; will attempt to use it 2024-03-12T01:58:42.079 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_583c7cd94fcfcfa9abcbbb46bafd029ba8504cb0/qa/tasks 2024-03-12T01:58:42.080 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-12T01:58:42.081 INFO:teuthology.task.internal:Checking packages... 2024-03-12T01:58:42.106 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'c2ea977b6956d1f694ccf66acc692fdd979e3baa' 2024-03-12T01:58:42.106 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-12T01:58:42.107 INFO:teuthology.packaging:ref: None 2024-03-12T01:58:42.107 INFO:teuthology.packaging:tag: None 2024-03-12T01:58:42.107 INFO:teuthology.packaging:branch: reef 2024-03-12T01:58:42.107 INFO:teuthology.packaging:sha1: c2ea977b6956d1f694ccf66acc692fdd979e3baa 2024-03-12T01:58:42.107 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=reef 2024-03-12T01:58:42.409 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-751-g62a11b3c-1focal 2024-03-12T01:58:42.410 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-12T01:58:42.417 INFO:teuthology.task.internal:no buildpackages task found 2024-03-12T01:58:42.417 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-12T01:58:42.461 INFO:teuthology.task.internal:Saving configuration 2024-03-12T01:58:42.473 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-12T01:58:42.478 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T01:58:42.503 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 01:52:04.059181', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-03-12T01:58:42.523 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 01:52:04.057746', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-03-12T01:58:42.523 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-12T01:58:42.529 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['host.a', 'client.0'] 2024-03-12T01:58:42.529 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['host.b', 'client.1'] 2024-03-12T01:58:42.529 INFO:teuthology.run_tasks:Running task console_log... 2024-03-12T01:58:42.600 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f899caa7280>, signals=[15]) 2024-03-12T01:58:42.601 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-12T01:58:42.606 INFO:teuthology.task.internal:Opening connections... 2024-03-12T01:58:42.606 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2024-03-12T01:58:42.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T01:58:42.708 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2024-03-12T01:58:42.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T01:58:42.799 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-12T01:58:42.805 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2024-03-12T01:58:42.812 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2024-03-12T01:58:42.813 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-03-12T01:58:42.859 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Ubuntu" 2024-03-12T01:58:42.859 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-03-12T01:58:42.859 INFO:teuthology.orchestra.run.smithi153.stdout:ID=ubuntu 2024-03-12T01:58:42.859 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE=debian 2024-03-12T01:58:42.859 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-03-12T01:58:42.859 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="20.04" 2024-03-12T01:58:42.860 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-12T01:58:42.860 INFO:teuthology.orchestra.run.smithi153.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-12T01:58:42.860 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-12T01:58:42.860 INFO:teuthology.orchestra.run.smithi153.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-12T01:58:42.860 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_CODENAME=focal 2024-03-12T01:58:42.860 INFO:teuthology.orchestra.run.smithi153.stdout:UBUNTU_CODENAME=focal 2024-03-12T01:58:42.860 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-03-12T01:58:42.882 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2024-03-12T01:58:42.888 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2024-03-12T01:58:42.888 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-03-12T01:58:42.894 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="Ubuntu" 2024-03-12T01:58:42.894 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-03-12T01:58:42.894 INFO:teuthology.orchestra.run.smithi163.stdout:ID=ubuntu 2024-03-12T01:58:42.894 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE=debian 2024-03-12T01:58:42.894 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-03-12T01:58:42.894 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="20.04" 2024-03-12T01:58:42.894 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-12T01:58:42.894 INFO:teuthology.orchestra.run.smithi163.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-12T01:58:42.894 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-12T01:58:42.894 INFO:teuthology.orchestra.run.smithi163.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-12T01:58:42.895 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_CODENAME=focal 2024-03-12T01:58:42.895 INFO:teuthology.orchestra.run.smithi163.stdout:UBUNTU_CODENAME=focal 2024-03-12T01:58:42.895 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-03-12T01:58:42.915 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-12T01:58:42.922 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-12T01:58:42.928 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-12T01:58:42.928 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2024-03-12T01:58:42.931 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2024-03-12T01:58:42.936 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-12T01:58:42.942 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-12T01:58:42.942 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2024-03-12T01:58:42.944 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2024-03-12T01:58:43.005 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-12T01:58:43.063 INFO:teuthology.run_tasks:Running task kernel... 2024-03-12T01:58:43.077 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-12T01:58:43.077 INFO:teuthology.task.kernel:normalize config orig: {'hwe': True} 2024-03-12T01:58:43.077 DEBUG:teuthology.task.kernel:normalized overrides {'host.a': {'hwe': True}, 'client.0': {'hwe': True}, 'host.b': {'hwe': True}, 'client.1': {'hwe': True}} 2024-03-12T01:58:43.078 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro', 'hwe': True}, 'host.b': {'kdb': True, 'sha1': 'distro', 'hwe': True}}, timeout 300 2024-03-12T01:58:43.078 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-12T01:58:43.078 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-12T01:58:43.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T01:58:43.084 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2024-03-12T01:58:43.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T01:58:43.085 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2024-03-12T01:58:43.091 INFO:teuthology.orchestra.run.smithi153.stdout:5.4.0-124-generic 2024-03-12T01:58:43.091 INFO:teuthology.task.kernel:Running kernel on smithi153: 5.4.0-124-generic 2024-03-12T01:58:43.091 DEBUG:teuthology.orchestra.run.smithi153:> sudo apt-get clean 2024-03-12T01:58:43.092 INFO:teuthology.orchestra.run.smithi163.stdout:5.4.0-124-generic 2024-03-12T01:58:43.092 INFO:teuthology.task.kernel:Running kernel on smithi163: 5.4.0-124-generic 2024-03-12T01:58:43.093 DEBUG:teuthology.orchestra.run.smithi163:> sudo apt-get clean 2024-03-12T01:58:43.192 DEBUG:teuthology.orchestra.run.smithi153:> sudo apt-get update 2024-03-12T01:58:43.250 DEBUG:teuthology.orchestra.run.smithi163:> sudo apt-get update 2024-03-12T01:58:43.385 INFO:teuthology.orchestra.run.smithi153.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-03-12T01:58:43.399 INFO:teuthology.orchestra.run.smithi153.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2024-03-12T01:58:43.426 INFO:teuthology.orchestra.run.smithi163.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2024-03-12T01:58:43.465 INFO:teuthology.orchestra.run.smithi153.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2024-03-12T01:58:43.483 INFO:teuthology.orchestra.run.smithi153.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2024-03-12T01:58:43.537 INFO:teuthology.orchestra.run.smithi163.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-03-12T01:58:43.619 INFO:teuthology.orchestra.run.smithi163.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2024-03-12T01:58:43.742 INFO:teuthology.orchestra.run.smithi163.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [719 kB] 2024-03-12T01:58:43.764 INFO:teuthology.orchestra.run.smithi153.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [944 kB] 2024-03-12T01:58:43.867 INFO:teuthology.orchestra.run.smithi163.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,774 kB] 2024-03-12T01:58:43.895 INFO:teuthology.orchestra.run.smithi153.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,153 kB] 2024-03-12T01:58:43.951 INFO:teuthology.orchestra.run.smithi163.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [421 kB] 2024-03-12T01:58:43.963 INFO:teuthology.orchestra.run.smithi163.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2024-03-12T01:58:43.964 INFO:teuthology.orchestra.run.smithi163.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,636 kB] 2024-03-12T01:58:43.974 INFO:teuthology.orchestra.run.smithi163.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2024-03-12T01:58:44.009 INFO:teuthology.orchestra.run.smithi153.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [504 kB] 2024-03-12T01:58:44.030 INFO:teuthology.orchestra.run.smithi153.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2024-03-12T01:58:44.031 INFO:teuthology.orchestra.run.smithi153.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [37.3 kB] 2024-03-12T01:58:44.039 INFO:teuthology.orchestra.run.smithi163.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [36.0 kB] 2024-03-12T01:58:44.040 INFO:teuthology.orchestra.run.smithi163.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [368 kB] 2024-03-12T01:58:44.040 INFO:teuthology.orchestra.run.smithi153.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,755 kB] 2024-03-12T01:58:44.050 INFO:teuthology.orchestra.run.smithi163.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2024-03-12T01:58:44.050 INFO:teuthology.orchestra.run.smithi163.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [653 kB] 2024-03-12T01:58:44.066 INFO:teuthology.orchestra.run.smithi163.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [948 kB] 2024-03-12T01:58:44.084 INFO:teuthology.orchestra.run.smithi153.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [719 kB] 2024-03-12T01:58:44.095 INFO:teuthology.orchestra.run.smithi163.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [199 kB] 2024-03-12T01:58:44.098 INFO:teuthology.orchestra.run.smithi163.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-03-12T01:58:44.099 INFO:teuthology.orchestra.run.smithi163.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,200 B] 2024-03-12T01:58:44.099 INFO:teuthology.orchestra.run.smithi163.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.9 kB] 2024-03-12T01:58:44.114 INFO:teuthology.orchestra.run.smithi153.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [385 kB] 2024-03-12T01:58:44.115 INFO:teuthology.orchestra.run.smithi163.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,153 kB] 2024-03-12T01:58:44.121 INFO:teuthology.orchestra.run.smithi163.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,796 B] 2024-03-12T01:58:44.121 INFO:teuthology.orchestra.run.smithi163.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2024-03-12T01:58:44.122 INFO:teuthology.orchestra.run.smithi153.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2024-03-12T01:58:44.122 INFO:teuthology.orchestra.run.smithi153.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,174 kB] 2024-03-12T01:58:44.154 INFO:teuthology.orchestra.run.smithi153.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [780 kB] 2024-03-12T01:58:44.167 INFO:teuthology.orchestra.run.smithi153.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [281 kB] 2024-03-12T01:58:44.175 INFO:teuthology.orchestra.run.smithi153.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2024-03-12T01:58:44.175 INFO:teuthology.orchestra.run.smithi153.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,444 B] 2024-03-12T01:58:44.175 INFO:teuthology.orchestra.run.smithi153.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [26.1 kB] 2024-03-12T01:58:44.178 INFO:teuthology.orchestra.run.smithi153.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,768 B] 2024-03-12T01:58:44.180 INFO:teuthology.orchestra.run.smithi153.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2024-03-12T01:58:44.206 INFO:teuthology.orchestra.run.smithi153.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-03-12T01:58:44.207 INFO:teuthology.orchestra.run.smithi153.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-03-12T01:58:44.208 INFO:teuthology.orchestra.run.smithi153.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-03-12T01:58:44.208 INFO:teuthology.orchestra.run.smithi153.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-03-12T01:58:44.226 INFO:teuthology.orchestra.run.smithi153.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-03-12T01:58:44.227 INFO:teuthology.orchestra.run.smithi153.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-03-12T01:58:44.421 INFO:teuthology.orchestra.run.smithi153.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,774 kB] 2024-03-12T01:58:44.508 INFO:teuthology.orchestra.run.smithi163.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [944 kB] 2024-03-12T01:58:44.536 INFO:teuthology.orchestra.run.smithi163.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [504 kB] 2024-03-12T01:58:44.549 INFO:teuthology.orchestra.run.smithi163.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2024-03-12T01:58:44.550 INFO:teuthology.orchestra.run.smithi163.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,755 kB] 2024-03-12T01:58:44.613 INFO:teuthology.orchestra.run.smithi153.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [421 kB] 2024-03-12T01:58:44.626 INFO:teuthology.orchestra.run.smithi153.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2024-03-12T01:58:44.627 INFO:teuthology.orchestra.run.smithi153.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [36.0 kB] 2024-03-12T01:58:44.628 INFO:teuthology.orchestra.run.smithi153.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,636 kB] 2024-03-12T01:58:44.633 INFO:teuthology.orchestra.run.smithi163.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [37.3 kB] 2024-03-12T01:58:44.634 INFO:teuthology.orchestra.run.smithi163.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [385 kB] 2024-03-12T01:58:44.640 INFO:teuthology.orchestra.run.smithi163.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2024-03-12T01:58:44.702 INFO:teuthology.orchestra.run.smithi153.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [368 kB] 2024-03-12T01:58:44.707 INFO:teuthology.orchestra.run.smithi153.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2024-03-12T01:58:44.707 INFO:teuthology.orchestra.run.smithi153.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [948 kB] 2024-03-12T01:58:44.746 INFO:teuthology.orchestra.run.smithi153.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [653 kB] 2024-03-12T01:58:44.805 INFO:teuthology.orchestra.run.smithi163.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [780 kB] 2024-03-12T01:58:44.920 INFO:teuthology.orchestra.run.smithi153.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [199 kB] 2024-03-12T01:58:45.353 INFO:teuthology.orchestra.run.smithi153.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-03-12T01:58:45.358 INFO:teuthology.orchestra.run.smithi153.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.9 kB] 2024-03-12T01:58:45.365 INFO:teuthology.orchestra.run.smithi153.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,200 B] 2024-03-12T01:58:45.367 INFO:teuthology.orchestra.run.smithi153.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,796 B] 2024-03-12T01:58:45.369 INFO:teuthology.orchestra.run.smithi153.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2024-03-12T01:58:45.402 INFO:teuthology.orchestra.run.smithi163.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,174 kB] 2024-03-12T01:58:45.512 INFO:teuthology.orchestra.run.smithi163.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [281 kB] 2024-03-12T01:58:45.533 INFO:teuthology.orchestra.run.smithi163.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2024-03-12T01:58:45.534 INFO:teuthology.orchestra.run.smithi163.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [26.1 kB] 2024-03-12T01:58:45.536 INFO:teuthology.orchestra.run.smithi163.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,444 B] 2024-03-12T01:58:45.536 INFO:teuthology.orchestra.run.smithi163.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,768 B] 2024-03-12T01:58:45.537 INFO:teuthology.orchestra.run.smithi163.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2024-03-12T01:58:45.537 INFO:teuthology.orchestra.run.smithi163.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-03-12T01:58:45.540 INFO:teuthology.orchestra.run.smithi163.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-03-12T01:58:45.542 INFO:teuthology.orchestra.run.smithi163.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-03-12T01:58:45.706 INFO:teuthology.orchestra.run.smithi163.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-03-12T01:58:45.892 INFO:teuthology.orchestra.run.smithi163.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-03-12T01:58:45.911 INFO:teuthology.orchestra.run.smithi163.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-03-12T01:58:47.961 INFO:teuthology.orchestra.run.smithi163.stdout:Fetched 19.4 MB in 3s (7,122 kB/s) 2024-03-12T01:58:48.116 INFO:teuthology.orchestra.run.smithi153.stdout:Fetched 19.4 MB in 3s (6,867 kB/s) 2024-03-12T01:58:49.456 INFO:teuthology.orchestra.run.smithi163.stdout:Reading package lists... 2024-03-12T01:58:49.481 DEBUG:teuthology.orchestra.run.smithi163:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2024-03-12T01:58:49.481 INFO:teuthology.orchestra.run.smithi153.stdout:Reading package lists... 2024-03-12T01:58:49.507 DEBUG:teuthology.orchestra.run.smithi153:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2024-03-12T01:58:49.565 INFO:teuthology.orchestra.run.smithi163.stdout:Reading package lists... 2024-03-12T01:58:49.585 INFO:teuthology.orchestra.run.smithi153.stdout:Reading package lists... 2024-03-12T01:58:49.754 INFO:teuthology.orchestra.run.smithi163.stdout:Building dependency tree... 2024-03-12T01:58:49.755 INFO:teuthology.orchestra.run.smithi163.stdout:Reading state information... 2024-03-12T01:58:49.796 INFO:teuthology.orchestra.run.smithi153.stdout:Building dependency tree... 2024-03-12T01:58:49.796 INFO:teuthology.orchestra.run.smithi153.stdout:Reading state information... 2024-03-12T01:58:49.929 INFO:teuthology.orchestra.run.smithi163.stdout:The following additional packages will be installed: 2024-03-12T01:58:49.929 INFO:teuthology.orchestra.run.smithi163.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2024-03-12T01:58:49.929 INFO:teuthology.orchestra.run.smithi163.stdout: libusbmuxd6 linux-image-5.15.0-100-generic linux-modules-5.15.0-100-generic 2024-03-12T01:58:49.929 INFO:teuthology.orchestra.run.smithi163.stdout: linux-modules-extra-5.15.0-100-generic thermald upower usbmuxd 2024-03-12T01:58:49.930 INFO:teuthology.orchestra.run.smithi163.stdout:Suggested packages: 2024-03-12T01:58:49.930 INFO:teuthology.orchestra.run.smithi163.stdout: libusbmuxd-tools fdutils linux-doc | linux-hwe-5.15-source-5.15.0 2024-03-12T01:58:49.930 INFO:teuthology.orchestra.run.smithi163.stdout: linux-hwe-5.15-tools linux-headers-5.15.0-100-generic 2024-03-12T01:58:49.984 INFO:teuthology.orchestra.run.smithi163.stdout:The following NEW packages will be installed: 2024-03-12T01:58:49.985 INFO:teuthology.orchestra.run.smithi163.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2024-03-12T01:58:49.985 INFO:teuthology.orchestra.run.smithi163.stdout: libusbmuxd6 linux-image-5.15.0-100-generic linux-image-generic-hwe-20.04 2024-03-12T01:58:49.985 INFO:teuthology.orchestra.run.smithi163.stdout: linux-modules-5.15.0-100-generic linux-modules-extra-5.15.0-100-generic 2024-03-12T01:58:49.985 INFO:teuthology.orchestra.run.smithi163.stdout: thermald upower usbmuxd 2024-03-12T01:58:49.989 INFO:teuthology.orchestra.run.smithi153.stdout:The following additional packages will be installed: 2024-03-12T01:58:49.990 INFO:teuthology.orchestra.run.smithi153.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2024-03-12T01:58:49.990 INFO:teuthology.orchestra.run.smithi153.stdout: libusbmuxd6 linux-image-5.15.0-100-generic linux-modules-5.15.0-100-generic 2024-03-12T01:58:49.990 INFO:teuthology.orchestra.run.smithi153.stdout: linux-modules-extra-5.15.0-100-generic thermald upower usbmuxd 2024-03-12T01:58:49.991 INFO:teuthology.orchestra.run.smithi153.stdout:Suggested packages: 2024-03-12T01:58:49.992 INFO:teuthology.orchestra.run.smithi153.stdout: libusbmuxd-tools fdutils linux-doc | linux-hwe-5.15-source-5.15.0 2024-03-12T01:58:49.992 INFO:teuthology.orchestra.run.smithi153.stdout: linux-hwe-5.15-tools linux-headers-5.15.0-100-generic 2024-03-12T01:58:50.041 INFO:teuthology.orchestra.run.smithi153.stdout:The following NEW packages will be installed: 2024-03-12T01:58:50.042 INFO:teuthology.orchestra.run.smithi153.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2024-03-12T01:58:50.042 INFO:teuthology.orchestra.run.smithi153.stdout: libusbmuxd6 linux-image-5.15.0-100-generic linux-image-generic-hwe-20.04 2024-03-12T01:58:50.043 INFO:teuthology.orchestra.run.smithi153.stdout: linux-modules-5.15.0-100-generic linux-modules-extra-5.15.0-100-generic 2024-03-12T01:58:50.043 INFO:teuthology.orchestra.run.smithi153.stdout: thermald upower usbmuxd 2024-03-12T01:58:50.049 INFO:teuthology.orchestra.run.smithi163.stdout:0 upgraded, 13 newly installed, 0 to remove and 291 not upgraded. 2024-03-12T01:58:50.049 INFO:teuthology.orchestra.run.smithi163.stdout:Need to get 90.9 MB of archives. 2024-03-12T01:58:50.050 INFO:teuthology.orchestra.run.smithi163.stdout:After this operation, 497 MB of additional disk space will be used. 2024-03-12T01:58:50.050 INFO:teuthology.orchestra.run.smithi163.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libdbus-glib-1-2 amd64 0.110-5fakssync1 [59.1 kB] 2024-03-12T01:58:50.101 INFO:teuthology.orchestra.run.smithi153.stdout:0 upgraded, 13 newly installed, 0 to remove and 291 not upgraded. 2024-03-12T01:58:50.101 INFO:teuthology.orchestra.run.smithi153.stdout:Need to get 90.9 MB of archives. 2024-03-12T01:58:50.101 INFO:teuthology.orchestra.run.smithi153.stdout:After this operation, 497 MB of additional disk space will be used. 2024-03-12T01:58:50.101 INFO:teuthology.orchestra.run.smithi153.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libdbus-glib-1-2 amd64 0.110-5fakssync1 [59.1 kB] 2024-03-12T01:58:50.120 INFO:teuthology.orchestra.run.smithi163.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 libplist3 amd64 2.1.0-4build2 [31.6 kB] 2024-03-12T01:58:50.127 INFO:teuthology.orchestra.run.smithi163.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 libusbmuxd6 amd64 2.0.1-2 [19.1 kB] 2024-03-12T01:58:50.132 INFO:teuthology.orchestra.run.smithi163.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 libimobiledevice6 amd64 1.2.1~git20191129.9f79242-1build1 [65.2 kB] 2024-03-12T01:58:50.150 INFO:teuthology.orchestra.run.smithi163.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 libupower-glib3 amd64 0.99.11-1build2 [43.2 kB] 2024-03-12T01:58:50.159 INFO:teuthology.orchestra.run.smithi163.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.15.0-100-generic amd64 5.15.0-100.110~20.04.1 [21.0 MB] 2024-03-12T01:58:50.167 INFO:teuthology.orchestra.run.smithi153.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 libplist3 amd64 2.1.0-4build2 [31.6 kB] 2024-03-12T01:58:50.174 INFO:teuthology.orchestra.run.smithi153.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 libusbmuxd6 amd64 2.0.1-2 [19.1 kB] 2024-03-12T01:58:50.178 INFO:teuthology.orchestra.run.smithi153.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 libimobiledevice6 amd64 1.2.1~git20191129.9f79242-1build1 [65.2 kB] 2024-03-12T01:58:50.196 INFO:teuthology.orchestra.run.smithi153.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 libupower-glib3 amd64 0.99.11-1build2 [43.2 kB] 2024-03-12T01:58:50.204 INFO:teuthology.orchestra.run.smithi153.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.15.0-100-generic amd64 5.15.0-100.110~20.04.1 [21.0 MB] 2024-03-12T01:58:50.612 INFO:teuthology.orchestra.run.smithi163.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.15.0-100-generic amd64 5.15.0-100.110~20.04.1 [11.4 MB] 2024-03-12T01:58:50.696 INFO:teuthology.orchestra.run.smithi153.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.15.0-100-generic amd64 5.15.0-100.110~20.04.1 [11.4 MB] 2024-03-12T01:58:50.745 INFO:teuthology.orchestra.run.smithi163.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.15.0-100-generic amd64 5.15.0-100.110~20.04.1 [57.9 MB] 2024-03-12T01:58:50.868 INFO:teuthology.orchestra.run.smithi153.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.15.0-100-generic amd64 5.15.0-100.110~20.04.1 [57.9 MB] 2024-03-12T01:58:52.080 INFO:teuthology.orchestra.run.smithi163.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic-hwe-20.04 amd64 5.15.0.100.110~20.04.52 [2,592 B] 2024-03-12T01:58:52.081 INFO:teuthology.orchestra.run.smithi163.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libevdev2 amd64 1.9.0+dfsg-1ubuntu0.2 [31.6 kB] 2024-03-12T01:58:52.081 INFO:teuthology.orchestra.run.smithi163.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 thermald amd64 1.9.1-1ubuntu0.6 [233 kB] 2024-03-12T01:58:52.087 INFO:teuthology.orchestra.run.smithi163.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 upower amd64 0.99.11-1build2 [104 kB] 2024-03-12T01:58:52.089 INFO:teuthology.orchestra.run.smithi163.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 usbmuxd amd64 1.1.1~git20191130.9af2b12-1 [38.4 kB] 2024-03-12T01:58:52.090 INFO:teuthology.orchestra.run.smithi153.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic-hwe-20.04 amd64 5.15.0.100.110~20.04.52 [2,592 B] 2024-03-12T01:58:52.163 INFO:teuthology.orchestra.run.smithi153.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libevdev2 amd64 1.9.0+dfsg-1ubuntu0.2 [31.6 kB] 2024-03-12T01:58:52.164 INFO:teuthology.orchestra.run.smithi153.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 thermald amd64 1.9.1-1ubuntu0.6 [233 kB] 2024-03-12T01:58:52.170 INFO:teuthology.orchestra.run.smithi153.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 upower amd64 0.99.11-1build2 [104 kB] 2024-03-12T01:58:52.172 INFO:teuthology.orchestra.run.smithi153.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 usbmuxd amd64 1.1.1~git20191130.9af2b12-1 [38.4 kB] 2024-03-12T01:58:52.821 INFO:teuthology.orchestra.run.smithi163.stdout:Fetched 90.9 MB in 2s (43.4 MB/s) 2024-03-12T01:58:52.839 INFO:teuthology.orchestra.run.smithi153.stdout:Fetched 90.9 MB in 2s (42.9 MB/s) 2024-03-12T01:58:53.052 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package libdbus-glib-1-2:amd64. 2024-03-12T01:58:53.135 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package libdbus-glib-1-2:amd64. 2024-03-12T01:58:54.810 INFO:teuthology.orchestra.run.smithi163.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-03-12T01:58:54.815 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../00-libdbus-glib-1-2_0.110-5fakssync1_amd64.deb ... 2024-03-12T01:58:54.836 INFO:teuthology.orchestra.run.smithi153.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-03-12T01:58:54.841 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../00-libdbus-glib-1-2_0.110-5fakssync1_amd64.deb ... 2024-03-12T01:58:54.982 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2024-03-12T01:58:55.016 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2024-03-12T01:58:55.352 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package libplist3:amd64. 2024-03-12T01:58:55.366 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../01-libplist3_2.1.0-4build2_amd64.deb ... 2024-03-12T01:58:55.386 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package libplist3:amd64. 2024-03-12T01:58:55.399 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../01-libplist3_2.1.0-4build2_amd64.deb ... 2024-03-12T01:58:55.410 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking libplist3:amd64 (2.1.0-4build2) ... 2024-03-12T01:58:55.436 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking libplist3:amd64 (2.1.0-4build2) ... 2024-03-12T01:58:55.772 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package libusbmuxd6:amd64. 2024-03-12T01:58:55.786 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../02-libusbmuxd6_2.0.1-2_amd64.deb ... 2024-03-12T01:58:55.797 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package libusbmuxd6:amd64. 2024-03-12T01:58:55.811 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../02-libusbmuxd6_2.0.1-2_amd64.deb ... 2024-03-12T01:58:55.830 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking libusbmuxd6:amd64 (2.0.1-2) ... 2024-03-12T01:58:55.855 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking libusbmuxd6:amd64 (2.0.1-2) ... 2024-03-12T01:58:56.158 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package libimobiledevice6:amd64. 2024-03-12T01:58:56.171 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../03-libimobiledevice6_1.2.1~git20191129.9f79242-1build1_amd64.deb ... 2024-03-12T01:58:56.226 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2024-03-12T01:58:56.227 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package libimobiledevice6:amd64. 2024-03-12T01:58:56.244 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../03-libimobiledevice6_1.2.1~git20191129.9f79242-1build1_amd64.deb ... 2024-03-12T01:58:56.291 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2024-03-12T01:58:56.561 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package libupower-glib3:amd64. 2024-03-12T01:58:56.574 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../04-libupower-glib3_0.99.11-1build2_amd64.deb ... 2024-03-12T01:58:56.621 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking libupower-glib3:amd64 (0.99.11-1build2) ... 2024-03-12T01:58:56.653 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package libupower-glib3:amd64. 2024-03-12T01:58:56.667 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../04-libupower-glib3_0.99.11-1build2_amd64.deb ... 2024-03-12T01:58:56.702 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking libupower-glib3:amd64 (0.99.11-1build2) ... 2024-03-12T01:58:56.972 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package linux-modules-5.15.0-100-generic. 2024-03-12T01:58:56.986 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../05-linux-modules-5.15.0-100-generic_5.15.0-100.110~20.04.1_amd64.deb ... 2024-03-12T01:58:57.030 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking linux-modules-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:58:57.064 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package linux-modules-5.15.0-100-generic. 2024-03-12T01:58:57.080 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../05-linux-modules-5.15.0-100-generic_5.15.0-100.110~20.04.1_amd64.deb ... 2024-03-12T01:58:57.174 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking linux-modules-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:58:59.531 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package linux-image-5.15.0-100-generic. 2024-03-12T01:58:59.546 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../06-linux-image-5.15.0-100-generic_5.15.0-100.110~20.04.1_amd64.deb ... 2024-03-12T01:58:59.663 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking linux-image-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:58:59.797 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package linux-image-5.15.0-100-generic. 2024-03-12T01:58:59.812 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../06-linux-image-5.15.0-100-generic_5.15.0-100.110~20.04.1_amd64.deb ... 2024-03-12T01:58:59.910 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking linux-image-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:59:00.143 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-100-generic. 2024-03-12T01:59:00.157 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../07-linux-modules-extra-5.15.0-100-generic_5.15.0-100.110~20.04.1_amd64.deb ... 2024-03-12T01:59:00.192 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking linux-modules-extra-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:59:00.433 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-100-generic. 2024-03-12T01:59:00.447 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../07-linux-modules-extra-5.15.0-100-generic_5.15.0-100.110~20.04.1_amd64.deb ... 2024-03-12T01:59:00.512 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking linux-modules-extra-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:59:06.895 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package linux-image-generic-hwe-20.04. 2024-03-12T01:59:06.912 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../08-linux-image-generic-hwe-20.04_5.15.0.100.110~20.04.52_amd64.deb ... 2024-03-12T01:59:06.952 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking linux-image-generic-hwe-20.04 (5.15.0.100.110~20.04.52) ... 2024-03-12T01:59:07.282 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package libevdev2:amd64. 2024-03-12T01:59:07.297 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../09-libevdev2_1.9.0+dfsg-1ubuntu0.2_amd64.deb ... 2024-03-12T01:59:07.338 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2024-03-12T01:59:07.538 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package linux-image-generic-hwe-20.04. 2024-03-12T01:59:07.554 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../08-linux-image-generic-hwe-20.04_5.15.0.100.110~20.04.52_amd64.deb ... 2024-03-12T01:59:07.596 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking linux-image-generic-hwe-20.04 (5.15.0.100.110~20.04.52) ... 2024-03-12T01:59:07.675 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package thermald. 2024-03-12T01:59:07.692 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../10-thermald_1.9.1-1ubuntu0.6_amd64.deb ... 2024-03-12T01:59:07.733 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking thermald (1.9.1-1ubuntu0.6) ... 2024-03-12T01:59:07.966 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package libevdev2:amd64. 2024-03-12T01:59:07.980 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../09-libevdev2_1.9.0+dfsg-1ubuntu0.2_amd64.deb ... 2024-03-12T01:59:08.024 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2024-03-12T01:59:08.280 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package upower. 2024-03-12T01:59:08.295 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../11-upower_0.99.11-1build2_amd64.deb ... 2024-03-12T01:59:08.356 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking upower (0.99.11-1build2) ... 2024-03-12T01:59:08.428 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package thermald. 2024-03-12T01:59:08.443 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../10-thermald_1.9.1-1ubuntu0.6_amd64.deb ... 2024-03-12T01:59:08.486 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking thermald (1.9.1-1ubuntu0.6) ... 2024-03-12T01:59:08.775 INFO:teuthology.orchestra.run.smithi153.stdout:Selecting previously unselected package usbmuxd. 2024-03-12T01:59:08.790 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing to unpack .../12-usbmuxd_1.1.1~git20191130.9af2b12-1_amd64.deb ... 2024-03-12T01:59:08.833 INFO:teuthology.orchestra.run.smithi153.stdout:Unpacking usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2024-03-12T01:59:09.075 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package upower. 2024-03-12T01:59:09.088 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../11-upower_0.99.11-1build2_amd64.deb ... 2024-03-12T01:59:09.140 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up libplist3:amd64 (2.1.0-4build2) ... 2024-03-12T01:59:09.150 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking upower (0.99.11-1build2) ... 2024-03-12T01:59:09.271 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2024-03-12T01:59:09.398 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up libusbmuxd6:amd64 (2.0.1-2) ... 2024-03-12T01:59:09.523 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up libupower-glib3:amd64 (0.99.11-1build2) ... 2024-03-12T01:59:09.580 INFO:teuthology.orchestra.run.smithi163.stdout:Selecting previously unselected package usbmuxd. 2024-03-12T01:59:09.593 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing to unpack .../12-usbmuxd_1.1.1~git20191130.9af2b12-1_amd64.deb ... 2024-03-12T01:59:09.630 INFO:teuthology.orchestra.run.smithi163.stdout:Unpacking usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2024-03-12T01:59:09.649 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2024-03-12T01:59:09.775 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2024-03-12T01:59:09.918 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up upower (0.99.11-1build2) ... 2024-03-12T01:59:09.953 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up libplist3:amd64 (2.1.0-4build2) ... 2024-03-12T01:59:10.076 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2024-03-12T01:59:10.236 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up libusbmuxd6:amd64 (2.0.1-2) ... 2024-03-12T01:59:10.379 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up libupower-glib3:amd64 (0.99.11-1build2) ... 2024-03-12T01:59:10.467 INFO:teuthology.orchestra.run.smithi153.stdout:upower.service is a disabled or a static unit, not starting it. 2024-03-12T01:59:10.530 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2024-03-12T01:59:10.539 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2024-03-12T01:59:10.656 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2024-03-12T01:59:10.711 INFO:teuthology.orchestra.run.smithi153.stdout:Warning: The home dir /var/lib/usbmux you specified can't be accessed: No such file or directory 2024-03-12T01:59:10.717 INFO:teuthology.orchestra.run.smithi153.stdout:Adding system user `usbmux' (UID 114) ... 2024-03-12T01:59:10.718 INFO:teuthology.orchestra.run.smithi153.stdout:Adding new user `usbmux' (UID 114) with group `plugdev' ... 2024-03-12T01:59:10.791 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up upower (0.99.11-1build2) ... 2024-03-12T01:59:11.198 INFO:teuthology.orchestra.run.smithi153.stdout:Not creating home directory `/var/lib/usbmux'. 2024-03-12T01:59:11.357 INFO:teuthology.orchestra.run.smithi163.stdout:upower.service is a disabled or a static unit, not starting it. 2024-03-12T01:59:11.401 INFO:teuthology.orchestra.run.smithi153.stdout:usbmuxd.service is a disabled or a static unit, not starting it. 2024-03-12T01:59:11.412 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2024-03-12T01:59:11.453 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up thermald (1.9.1-1ubuntu0.6) ... 2024-03-12T01:59:11.638 INFO:teuthology.orchestra.run.smithi163.stdout:Warning: The home dir /var/lib/usbmux you specified can't be accessed: No such file or directory 2024-03-12T01:59:11.652 INFO:teuthology.orchestra.run.smithi163.stdout:Adding system user `usbmux' (UID 114) ... 2024-03-12T01:59:11.653 INFO:teuthology.orchestra.run.smithi163.stdout:Adding new user `usbmux' (UID 114) with group `plugdev' ... 2024-03-12T01:59:11.764 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.thermald.service -> /lib/systemd/system/thermald.service. 2024-03-12T01:59:11.764 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/thermald.service -> /lib/systemd/system/thermald.service. 2024-03-12T01:59:12.216 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up linux-image-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:59:12.217 INFO:teuthology.orchestra.run.smithi163.stdout:Not creating home directory `/var/lib/usbmux'. 2024-03-12T01:59:12.432 INFO:teuthology.orchestra.run.smithi163.stdout:usbmuxd.service is a disabled or a static unit, not starting it. 2024-03-12T01:59:12.501 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up thermald (1.9.1-1ubuntu0.6) ... 2024-03-12T01:59:12.814 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.thermald.service -> /lib/systemd/system/thermald.service. 2024-03-12T01:59:12.814 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/thermald.service -> /lib/systemd/system/thermald.service. 2024-03-12T01:59:13.273 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up linux-image-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:59:13.607 INFO:teuthology.orchestra.run.smithi153.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-100-generic 2024-03-12T01:59:13.607 INFO:teuthology.orchestra.run.smithi153.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-100-generic 2024-03-12T01:59:13.951 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up linux-modules-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:59:14.591 INFO:teuthology.orchestra.run.smithi163.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-100-generic 2024-03-12T01:59:14.591 INFO:teuthology.orchestra.run.smithi163.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-100-generic 2024-03-12T01:59:14.776 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up linux-modules-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:59:15.344 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up linux-modules-extra-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:59:16.161 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up linux-modules-extra-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:59:16.731 INFO:teuthology.orchestra.run.smithi153.stdout:Setting up linux-image-generic-hwe-20.04 (5.15.0.100.110~20.04.52) ... 2024-03-12T01:59:16.856 INFO:teuthology.orchestra.run.smithi153.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-03-12T01:59:17.137 INFO:teuthology.orchestra.run.smithi153.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-03-12T01:59:17.563 INFO:teuthology.orchestra.run.smithi163.stdout:Setting up linux-image-generic-hwe-20.04 (5.15.0.100.110~20.04.52) ... 2024-03-12T01:59:17.714 INFO:teuthology.orchestra.run.smithi163.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-03-12T01:59:17.925 INFO:teuthology.orchestra.run.smithi153.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-03-12T01:59:18.023 INFO:teuthology.orchestra.run.smithi153.stdout:Processing triggers for linux-image-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:59:18.087 INFO:teuthology.orchestra.run.smithi153.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-12T01:59:18.088 INFO:teuthology.orchestra.run.smithi153.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-100-generic 2024-03-12T01:59:19.104 INFO:teuthology.orchestra.run.smithi163.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-03-12T01:59:19.941 INFO:teuthology.orchestra.run.smithi163.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-03-12T01:59:20.049 INFO:teuthology.orchestra.run.smithi163.stdout:Processing triggers for linux-image-5.15.0-100-generic (5.15.0-100.110~20.04.1) ... 2024-03-12T01:59:20.139 INFO:teuthology.orchestra.run.smithi163.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-12T01:59:20.139 INFO:teuthology.orchestra.run.smithi163.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-100-generic 2024-03-12T01:59:43.517 INFO:teuthology.orchestra.run.smithi153.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-12T01:59:43.517 INFO:teuthology.orchestra.run.smithi153.stdout:Sourcing file `/etc/default/grub' 2024-03-12T01:59:43.531 INFO:teuthology.orchestra.run.smithi153.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-12T01:59:43.548 INFO:teuthology.orchestra.run.smithi153.stdout:Generating grub configuration file ... 2024-03-12T01:59:43.886 INFO:teuthology.orchestra.run.smithi153.stdout:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-12T01:59:43.905 INFO:teuthology.orchestra.run.smithi153.stdout:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-12T01:59:44.279 INFO:teuthology.orchestra.run.smithi153.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-03-12T01:59:44.291 INFO:teuthology.orchestra.run.smithi153.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-03-12T01:59:45.777 INFO:teuthology.orchestra.run.smithi163.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-12T01:59:45.777 INFO:teuthology.orchestra.run.smithi163.stdout:Sourcing file `/etc/default/grub' 2024-03-12T01:59:45.790 INFO:teuthology.orchestra.run.smithi163.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-12T01:59:45.821 INFO:teuthology.orchestra.run.smithi163.stdout:Generating grub configuration file ... 2024-03-12T01:59:46.168 INFO:teuthology.orchestra.run.smithi163.stdout:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-12T01:59:46.187 INFO:teuthology.orchestra.run.smithi163.stdout:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-12T01:59:46.227 INFO:teuthology.orchestra.run.smithi153.stdout:done 2024-03-12T01:59:46.579 INFO:teuthology.orchestra.run.smithi163.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-03-12T01:59:46.591 INFO:teuthology.orchestra.run.smithi163.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-03-12T01:59:46.666 DEBUG:teuthology.orchestra.run.smithi153:> dpkg -s linux-image-generic-hwe-20.04 2024-03-12T01:59:46.723 INFO:teuthology.orchestra.run.smithi153.stdout:Package: linux-image-generic-hwe-20.04 2024-03-12T01:59:46.723 INFO:teuthology.orchestra.run.smithi153.stdout:Status: install ok installed 2024-03-12T01:59:46.723 INFO:teuthology.orchestra.run.smithi153.stdout:Priority: optional 2024-03-12T01:59:46.723 INFO:teuthology.orchestra.run.smithi153.stdout:Section: kernel 2024-03-12T01:59:46.723 INFO:teuthology.orchestra.run.smithi153.stdout:Installed-Size: 20 2024-03-12T01:59:46.723 INFO:teuthology.orchestra.run.smithi153.stdout:Maintainer: Ubuntu Kernel Team 2024-03-12T01:59:46.723 INFO:teuthology.orchestra.run.smithi153.stdout:Architecture: amd64 2024-03-12T01:59:46.724 INFO:teuthology.orchestra.run.smithi153.stdout:Source: linux-meta-hwe-5.15 2024-03-12T01:59:46.724 INFO:teuthology.orchestra.run.smithi153.stdout:Version: 5.15.0.100.110~20.04.52 2024-03-12T01:59:46.724 INFO:teuthology.orchestra.run.smithi153.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-12T01:59:46.724 INFO:teuthology.orchestra.run.smithi153.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-12T01:59:46.724 INFO:teuthology.orchestra.run.smithi153.stdout:Recommends: thermald 2024-03-12T01:59:46.724 INFO:teuthology.orchestra.run.smithi153.stdout:Description: Generic Linux kernel image 2024-03-12T01:59:46.724 INFO:teuthology.orchestra.run.smithi153.stdout: This package will always depend on the latest generic kernel image 2024-03-12T01:59:46.724 INFO:teuthology.orchestra.run.smithi153.stdout: available. 2024-03-12T01:59:46.725 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.15.0-100-generic 2024-03-12T01:59:46.725 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-12T01:59:46.725 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-12T01:59:46.725 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-03-12T01:59:46.725 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi153.front.sepia.ceph.com, path=None, version=distro) 2024-03-12T01:59:46.725 DEBUG:teuthology.orchestra.run.smithi153:> sudo apt-get clean 2024-03-12T01:59:46.821 DEBUG:teuthology.orchestra.run.smithi153:> sudo apt-get update 2024-03-12T01:59:46.963 INFO:teuthology.orchestra.run.smithi153.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-03-12T01:59:46.972 INFO:teuthology.orchestra.run.smithi153.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-03-12T01:59:46.994 INFO:teuthology.orchestra.run.smithi153.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-03-12T01:59:46.995 INFO:teuthology.orchestra.run.smithi153.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-03-12T01:59:48.560 INFO:teuthology.orchestra.run.smithi163.stdout:done 2024-03-12T01:59:48.734 INFO:teuthology.orchestra.run.smithi153.stdout:Reading package lists... 2024-03-12T01:59:48.760 DEBUG:teuthology.orchestra.run.smithi153:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2024-03-12T01:59:48.830 INFO:teuthology.orchestra.run.smithi153.stdout:Reading package lists... 2024-03-12T01:59:49.029 INFO:teuthology.orchestra.run.smithi153.stdout:Building dependency tree... 2024-03-12T01:59:49.029 INFO:teuthology.orchestra.run.smithi153.stdout:Reading state information... 2024-03-12T01:59:49.043 DEBUG:teuthology.orchestra.run.smithi163:> dpkg -s linux-image-generic-hwe-20.04 2024-03-12T01:59:49.092 INFO:teuthology.orchestra.run.smithi163.stdout:Package: linux-image-generic-hwe-20.04 2024-03-12T01:59:49.092 INFO:teuthology.orchestra.run.smithi163.stdout:Status: install ok installed 2024-03-12T01:59:49.092 INFO:teuthology.orchestra.run.smithi163.stdout:Priority: optional 2024-03-12T01:59:49.092 INFO:teuthology.orchestra.run.smithi163.stdout:Section: kernel 2024-03-12T01:59:49.092 INFO:teuthology.orchestra.run.smithi163.stdout:Installed-Size: 20 2024-03-12T01:59:49.092 INFO:teuthology.orchestra.run.smithi163.stdout:Maintainer: Ubuntu Kernel Team 2024-03-12T01:59:49.093 INFO:teuthology.orchestra.run.smithi163.stdout:Architecture: amd64 2024-03-12T01:59:49.093 INFO:teuthology.orchestra.run.smithi163.stdout:Source: linux-meta-hwe-5.15 2024-03-12T01:59:49.093 INFO:teuthology.orchestra.run.smithi163.stdout:Version: 5.15.0.100.110~20.04.52 2024-03-12T01:59:49.093 INFO:teuthology.orchestra.run.smithi163.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-12T01:59:49.093 INFO:teuthology.orchestra.run.smithi163.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-12T01:59:49.093 INFO:teuthology.orchestra.run.smithi163.stdout:Recommends: thermald 2024-03-12T01:59:49.093 INFO:teuthology.orchestra.run.smithi163.stdout:Description: Generic Linux kernel image 2024-03-12T01:59:49.093 INFO:teuthology.orchestra.run.smithi163.stdout: This package will always depend on the latest generic kernel image 2024-03-12T01:59:49.093 INFO:teuthology.orchestra.run.smithi163.stdout: available. 2024-03-12T01:59:49.093 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.15.0-100-generic 2024-03-12T01:59:49.093 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-12T01:59:49.094 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-12T01:59:49.094 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-03-12T01:59:49.094 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi163.front.sepia.ceph.com, path=None, version=distro) 2024-03-12T01:59:49.094 DEBUG:teuthology.orchestra.run.smithi163:> sudo apt-get clean 2024-03-12T01:59:49.168 DEBUG:teuthology.orchestra.run.smithi163:> sudo apt-get update 2024-03-12T01:59:49.247 INFO:teuthology.orchestra.run.smithi153.stdout:linux-image-generic-hwe-20.04 is already the newest version (5.15.0.100.110~20.04.52). 2024-03-12T01:59:49.247 INFO:teuthology.orchestra.run.smithi153.stdout:0 upgraded, 0 newly installed, 0 to remove and 291 not upgraded. 2024-03-12T01:59:49.249 DEBUG:teuthology.orchestra.run.smithi153:> dpkg -s linux-image-generic-hwe-20.04 2024-03-12T01:59:49.270 INFO:teuthology.orchestra.run.smithi153.stdout:Package: linux-image-generic-hwe-20.04 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Status: install ok installed 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Priority: optional 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Section: kernel 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Installed-Size: 20 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Maintainer: Ubuntu Kernel Team 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Architecture: amd64 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Source: linux-meta-hwe-5.15 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Version: 5.15.0.100.110~20.04.52 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Recommends: thermald 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout:Description: Generic Linux kernel image 2024-03-12T01:59:49.271 INFO:teuthology.orchestra.run.smithi153.stdout: This package will always depend on the latest generic kernel image 2024-03-12T01:59:49.272 INFO:teuthology.orchestra.run.smithi153.stdout: available. 2024-03-12T01:59:49.272 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2024-03-12T01:59:49.276 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.2anCwWp6YQ 2024-03-12T01:59:49.276 DEBUG:teuthology.orchestra.run.smithi153:> sudo cp /boot/grub/grub.cfg /tmp/tmp.2anCwWp6YQ 2024-03-12T01:59:49.286 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.2anCwWp6YQ 2024-03-12T01:59:49.352 INFO:teuthology.orchestra.run.smithi163.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-03-12T01:59:49.359 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.2anCwWp6YQ is 9KB 2024-03-12T01:59:49.371 INFO:teuthology.orchestra.run.smithi163.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-03-12T01:59:49.375 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.2anCwWp6YQ 2024-03-12T01:59:49.382 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-12T01:59:49.396 INFO:teuthology.orchestra.run.smithi163.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-03-12T01:59:49.415 INFO:teuthology.orchestra.run.smithi163.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-03-12T01:59:49.436 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T01:59:49.436 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-12T01:59:49.436 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-12T01:59:49.462 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-100-generic 2024-03-12T01:59:49.462 DEBUG:teuthology.orchestra.run.smithi153:> sudo update-grub 2024-03-12T01:59:50.779 INFO:teuthology.orchestra.run.smithi153.stderr:Sourcing file `/etc/default/grub' 2024-03-12T01:59:50.788 INFO:teuthology.orchestra.run.smithi153.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-12T01:59:50.800 INFO:teuthology.orchestra.run.smithi153.stderr:Generating grub configuration file ... 2024-03-12T01:59:51.033 INFO:teuthology.orchestra.run.smithi153.stderr:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-12T01:59:51.045 INFO:teuthology.orchestra.run.smithi153.stderr:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-12T01:59:51.160 INFO:teuthology.orchestra.run.smithi163.stdout:Reading package lists... 2024-03-12T01:59:51.187 DEBUG:teuthology.orchestra.run.smithi163:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2024-03-12T01:59:51.255 INFO:teuthology.orchestra.run.smithi163.stdout:Reading package lists... 2024-03-12T01:59:51.442 INFO:teuthology.orchestra.run.smithi153.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-03-12T01:59:51.454 INFO:teuthology.orchestra.run.smithi153.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-03-12T01:59:51.462 INFO:teuthology.orchestra.run.smithi163.stdout:Building dependency tree... 2024-03-12T01:59:51.463 INFO:teuthology.orchestra.run.smithi163.stdout:Reading state information... 2024-03-12T01:59:51.686 INFO:teuthology.orchestra.run.smithi163.stdout:linux-image-generic-hwe-20.04 is already the newest version (5.15.0.100.110~20.04.52). 2024-03-12T01:59:51.686 INFO:teuthology.orchestra.run.smithi163.stdout:0 upgraded, 0 newly installed, 0 to remove and 291 not upgraded. 2024-03-12T01:59:51.688 DEBUG:teuthology.orchestra.run.smithi163:> dpkg -s linux-image-generic-hwe-20.04 2024-03-12T01:59:51.709 INFO:teuthology.orchestra.run.smithi163.stdout:Package: linux-image-generic-hwe-20.04 2024-03-12T01:59:51.709 INFO:teuthology.orchestra.run.smithi163.stdout:Status: install ok installed 2024-03-12T01:59:51.709 INFO:teuthology.orchestra.run.smithi163.stdout:Priority: optional 2024-03-12T01:59:51.709 INFO:teuthology.orchestra.run.smithi163.stdout:Section: kernel 2024-03-12T01:59:51.709 INFO:teuthology.orchestra.run.smithi163.stdout:Installed-Size: 20 2024-03-12T01:59:51.709 INFO:teuthology.orchestra.run.smithi163.stdout:Maintainer: Ubuntu Kernel Team 2024-03-12T01:59:51.709 INFO:teuthology.orchestra.run.smithi163.stdout:Architecture: amd64 2024-03-12T01:59:51.709 INFO:teuthology.orchestra.run.smithi163.stdout:Source: linux-meta-hwe-5.15 2024-03-12T01:59:51.709 INFO:teuthology.orchestra.run.smithi163.stdout:Version: 5.15.0.100.110~20.04.52 2024-03-12T01:59:51.710 INFO:teuthology.orchestra.run.smithi163.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-12T01:59:51.710 INFO:teuthology.orchestra.run.smithi163.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-12T01:59:51.710 INFO:teuthology.orchestra.run.smithi163.stdout:Recommends: thermald 2024-03-12T01:59:51.710 INFO:teuthology.orchestra.run.smithi163.stdout:Description: Generic Linux kernel image 2024-03-12T01:59:51.710 INFO:teuthology.orchestra.run.smithi163.stdout: This package will always depend on the latest generic kernel image 2024-03-12T01:59:51.710 INFO:teuthology.orchestra.run.smithi163.stdout: available. 2024-03-12T01:59:51.710 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-03-12T01:59:51.716 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.TxCI3mGV3A 2024-03-12T01:59:51.717 DEBUG:teuthology.orchestra.run.smithi163:> sudo cp /boot/grub/grub.cfg /tmp/tmp.TxCI3mGV3A 2024-03-12T01:59:51.726 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.TxCI3mGV3A 2024-03-12T01:59:51.819 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.TxCI3mGV3A is 9KB 2024-03-12T01:59:51.834 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.TxCI3mGV3A 2024-03-12T01:59:51.840 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-12T01:59:51.855 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T01:59:51.855 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-12T01:59:51.855 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-12T01:59:51.935 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-100-generic 2024-03-12T01:59:51.935 DEBUG:teuthology.orchestra.run.smithi163:> sudo update-grub 2024-03-12T01:59:52.473 INFO:teuthology.orchestra.run.smithi153.stderr:done 2024-03-12T01:59:52.474 DEBUG:teuthology.orchestra.run.smithi153:> sudo shutdown -r now 2024-03-12T01:59:53.365 INFO:teuthology.orchestra.run.smithi163.stderr:Sourcing file `/etc/default/grub' 2024-03-12T01:59:53.370 INFO:teuthology.orchestra.run.smithi163.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-12T01:59:53.386 INFO:teuthology.orchestra.run.smithi163.stderr:Generating grub configuration file ... 2024-03-12T01:59:53.642 INFO:teuthology.orchestra.run.smithi163.stderr:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-12T01:59:53.654 INFO:teuthology.orchestra.run.smithi163.stderr:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-12T01:59:53.993 INFO:teuthology.orchestra.run.smithi163.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-03-12T01:59:54.004 INFO:teuthology.orchestra.run.smithi163.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-03-12T01:59:54.980 INFO:teuthology.orchestra.run.smithi163.stderr:done 2024-03-12T01:59:54.982 DEBUG:teuthology.orchestra.run.smithi163:> sudo shutdown -r now 2024-03-12T02:00:22.502 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-12T02:00:22.503 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-12T02:00:22.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T02:00:24.986 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-12T02:00:24.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-12T02:00:24.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T02:00:40.993 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-03-12T02:00:43.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-12T02:00:49.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-12T02:00:49.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T02:00:52.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-12T02:00:52.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T02:00:53.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-03-12T02:00:55.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-12T02:01:05.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-12T02:01:05.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T02:01:07.617 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-12T02:01:07.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T02:01:10.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-12T02:01:25.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-12T02:01:25.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T02:01:44.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-12T02:02:02.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-12T02:02:02.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T02:02:02.482 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-03-12T02:02:03.020 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-12T02:02:03.020 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.15.0-100-generic"... 2024-03-12T02:02:03.020 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2024-03-12T02:02:03.067 INFO:teuthology.orchestra.run.smithi163.stdout:5.15.0-100-generic 2024-03-12T02:02:03.068 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-100-generic vs 5.15.0-100-generic 2024-03-12T02:02:03.068 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-12T02:02:03.068 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-12T02:02:04.069 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-12T02:02:04.069 DEBUG:teuthology.orchestra.run.smithi163:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-12T02:02:04.422 INFO:teuthology.orchestra.run.smithi163.stdout:ttyS1 2024-03-12T02:02:04.452 DEBUG:teuthology.parallel:result is None 2024-03-12T02:02:05.065 DEBUG:teuthology.orchestra.remote:timed out 2024-03-12T02:02:20.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-12T02:02:20.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T02:02:20.448 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-03-12T02:02:20.912 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-12T02:02:20.912 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.15.0-100-generic"... 2024-03-12T02:02:20.912 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2024-03-12T02:02:20.962 INFO:teuthology.orchestra.run.smithi153.stdout:5.15.0-100-generic 2024-03-12T02:02:20.962 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-100-generic vs 5.15.0-100-generic 2024-03-12T02:02:20.962 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-12T02:02:20.962 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-12T02:02:21.963 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-12T02:02:21.963 DEBUG:teuthology.orchestra.run.smithi153:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-12T02:02:22.333 INFO:teuthology.orchestra.run.smithi153.stdout:ttyS1 2024-03-12T02:02:22.366 DEBUG:teuthology.parallel:result is None 2024-03-12T02:02:22.366 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-12T02:02:22.375 INFO:teuthology.task.internal:Creating test directory... 2024-03-12T02:02:22.375 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-12T02:02:22.377 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-12T02:02:22.385 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-12T02:02:22.407 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-12T02:02:22.416 INFO:teuthology.task.internal:Creating archive directory... 2024-03-12T02:02:22.416 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-12T02:02:22.425 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-12T02:02:22.520 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-12T02:02:22.527 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-12T02:02:22.527 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-12T02:02:22.529 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-12T02:02:22.544 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T02:02:22.546 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T02:02:22.551 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T02:02:22.553 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-12T02:02:22.554 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-12T02:02:22.561 INFO:teuthology.task.internal:Configuring sudo... 2024-03-12T02:02:22.561 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-12T02:02:22.597 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-12T02:02:22.614 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-12T02:02:22.624 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-12T02:02:22.624 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-12T02:02:22.653 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-12T02:02:22.664 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T02:02:22.706 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T02:02:22.754 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:02:22.755 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-12T02:02:22.821 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-12T02:02:22.829 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-12T02:02:22.880 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T02:02:22.880 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-12T02:02:22.942 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2024-03-12T02:02:22.944 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2024-03-12T02:02:23.036 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-12T02:02:23.046 INFO:teuthology.task.internal:Starting timer... 2024-03-12T02:02:23.046 INFO:teuthology.run_tasks:Running task pcp... 2024-03-12T02:02:23.060 INFO:teuthology.run_tasks:Running task selinux... 2024-03-12T02:02:23.072 DEBUG:teuthology.task.selinux:Excluding smithi153: OS 'ubuntu' does not support SELinux 2024-03-12T02:02:23.072 DEBUG:teuthology.task.selinux:Excluding smithi163: OS 'ubuntu' does not support SELinux 2024-03-12T02:02:23.072 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-12T02:02:23.072 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-12T02:02:23.072 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-12T02:02:23.072 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-12T02:02:23.083 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-12T02:02:23.100 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-12T02:02:23.102 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi153.front.sepia.ceph.com,smithi163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-12T02:07:52.598 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi153.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com')] 2024-03-12T02:07:52.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-12T02:07:52.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T02:07:52.691 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-03-12T02:07:52.816 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-12T02:07:52.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-12T02:07:52.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T02:07:52.896 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-03-12T02:07:53.020 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-12T02:07:53.020 INFO:teuthology.run_tasks:Running task clock... 2024-03-12T02:07:53.032 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-12T02:07:53.032 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-12T02:07:53.032 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T02:07:53.035 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-12T02:07:53.035 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T02:07:53.064 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-03-12T02:07:53.064 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: Command line: ntpd -gq 2024-03-12T02:07:53.065 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: proto: precision = 0.083 usec (-23) 2024-03-12T02:07:53.065 INFO:teuthology.orchestra.run.smithi153.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-12T02:07:53.066 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-12T02:07:53.066 INFO:teuthology.orchestra.run.smithi153.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-12T02:07:53.067 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: restrict ::: KOD does nothing without LIMITED. 2024-03-12T02:07:53.067 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: Listen and drop on 0 v6wildcard [::]:123 2024-03-12T02:07:53.067 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-12T02:07:53.067 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: Listen normally on 2 lo 127.0.0.1:123 2024-03-12T02:07:53.067 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: Listen normally on 3 enp3s0f1 172.21.15.153:123 2024-03-12T02:07:53.067 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: Listen normally on 4 lo [::1]:123 2024-03-12T02:07:53.067 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6deb%5]:123 2024-03-12T02:07:53.067 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:07:53 ntpd[12766]: Listening on routing socket on fd #22 for interface updates 2024-03-12T02:07:53.093 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-03-12T02:07:53.093 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: Command line: ntpd -gq 2024-03-12T02:07:53.094 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: proto: precision = 0.064 usec (-24) 2024-03-12T02:07:53.094 INFO:teuthology.orchestra.run.smithi163.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-12T02:07:53.095 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-12T02:07:53.095 INFO:teuthology.orchestra.run.smithi163.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-12T02:07:53.096 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: restrict ::: KOD does nothing without LIMITED. 2024-03-12T02:07:53.096 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: Listen and drop on 0 v6wildcard [::]:123 2024-03-12T02:07:53.096 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-12T02:07:53.096 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: Listen normally on 2 lo 127.0.0.1:123 2024-03-12T02:07:53.096 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: Listen normally on 3 enp3s0f1 172.21.15.163:123 2024-03-12T02:07:53.096 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: Listen normally on 4 lo [::1]:123 2024-03-12T02:07:53.096 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7d5b%5]:123 2024-03-12T02:07:53.096 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:07:53 ntpd[12721]: Listening on routing socket on fd #22 for interface updates 2024-03-12T02:07:54.065 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:54 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:54.065 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:54 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:54.065 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:54 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:54.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:54 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:54.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:54 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:54.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:54 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:54.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:54 ntpd[12766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:54.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:54 ntpd[12766]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:56.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:56.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:56.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.068 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:56.068 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:56.068 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.068 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.068 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:56.068 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:56 ntpd[12766]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:56.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:56.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:56.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.097 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.097 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:56.097 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:56.097 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:56.097 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:56.097 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:56.097 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:56 ntpd[12721]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:58.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:58.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:07:58 ntpd[12766]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:58.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:07:58.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:07:58.097 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:07:58.097 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:07:58 ntpd[12721]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:08:00.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:00.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:00.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:00.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:00.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:00.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:00.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:00.066 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:00.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:08:00.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:08:00.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:00.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:00.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:08:00.067 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:08:00.067 INFO:teuthology.orchestra.run.smithi153.stdout:12 Mar 02:08:00 ntpd[12766]: ntpd: time slew +0.001027 s 2024-03-12T02:08:00.067 INFO:teuthology.orchestra.run.smithi153.stdout:ntpd: time slew +0.001027s 2024-03-12T02:08:00.068 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-12T02:08:00.068 INFO:teuthology.orchestra.run.smithi153.stderr:12 Mar 02:08:00 ntpd[12766]: can't open /var/log/ntpstats/loopstats.20240312: Permission denied 2024-03-12T02:08:00.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:00.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:00.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:00.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:08:00.097 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:00 ntpd[12721]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:08:00.128 INFO:teuthology.orchestra.run.smithi153.stdout: remote refid st t when poll reach delay offset jitter 2024-03-12T02:08:00.128 INFO:teuthology.orchestra.run.smithi153.stdout:============================================================================== 2024-03-12T02:08:00.129 INFO:teuthology.orchestra.run.smithi153.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-03-12T02:08:00.129 INFO:teuthology.orchestra.run.smithi153.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-03-12T02:08:00.129 INFO:teuthology.orchestra.run.smithi153.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-03-12T02:08:00.129 INFO:teuthology.orchestra.run.smithi153.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-03-12T02:08:02.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:02.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:02.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:02.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:02.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:02.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:02.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-12T02:08:02.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: can't open /var/log/ntpstats/rawstats.20240312: Permission denied 2024-03-12T02:08:02.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-12T02:08:02.095 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: can't open /var/log/ntpstats/peerstats.20240312: Permission denied 2024-03-12T02:08:02.096 INFO:teuthology.orchestra.run.smithi163.stdout:12 Mar 02:08:02 ntpd[12721]: ntpd: time slew -0.002896 s 2024-03-12T02:08:02.096 INFO:teuthology.orchestra.run.smithi163.stdout:ntpd: time slew -0.002896s 2024-03-12T02:08:02.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-12T02:08:02.096 INFO:teuthology.orchestra.run.smithi163.stderr:12 Mar 02:08:02 ntpd[12721]: can't open /var/log/ntpstats/loopstats.20240312: Permission denied 2024-03-12T02:08:02.151 INFO:teuthology.orchestra.run.smithi163.stdout: remote refid st t when poll reach delay offset jitter 2024-03-12T02:08:02.151 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================== 2024-03-12T02:08:02.151 INFO:teuthology.orchestra.run.smithi163.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-03-12T02:08:02.151 INFO:teuthology.orchestra.run.smithi163.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-03-12T02:08:02.151 INFO:teuthology.orchestra.run.smithi163.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-03-12T02:08:02.151 INFO:teuthology.orchestra.run.smithi163.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-03-12T02:08:02.152 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-03-12T02:08:02.162 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-03-12T02:08:02.162 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:08:02.162 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T02:08:02.169 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-12T02:08:02.170 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2024-03-12T02:08:02.220 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-12T02:08:02.221 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T02:08:02.221 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 489 Links: 1 2024-03-12T02:08:02.221 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T02:08:02.221 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-12 02:06:23.316400123 +0000 2024-03-12T02:08:02.221 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-12 02:06:23.052406208 +0000 2024-03-12T02:08:02.221 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-12 02:06:23.052406208 +0000 2024-03-12T02:08:02.221 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2024-03-12T02:08:02.221 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-12T02:08:02.276 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-12T02:08:02.276 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-12T02:08:02.276 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000421804 s, 1.2 MB/s 2024-03-12T02:08:02.277 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-12T02:08:02.326 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2024-03-12T02:08:02.376 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-12T02:08:02.377 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T02:08:02.377 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 495 Links: 1 2024-03-12T02:08:02.377 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T02:08:02.377 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-12 02:06:23.536395052 +0000 2024-03-12T02:08:02.377 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-12 02:06:23.532395144 +0000 2024-03-12T02:08:02.377 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-12 02:06:23.532395144 +0000 2024-03-12T02:08:02.377 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2024-03-12T02:08:02.377 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-12T02:08:02.432 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-12T02:08:02.432 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-12T02:08:02.432 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000442051 s, 1.2 MB/s 2024-03-12T02:08:02.433 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-12T02:08:02.486 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2024-03-12T02:08:02.536 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-12T02:08:02.537 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T02:08:02.537 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 502 Links: 1 2024-03-12T02:08:02.537 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T02:08:02.537 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-12 02:06:23.988384634 +0000 2024-03-12T02:08:02.537 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-12 02:06:23.984384725 +0000 2024-03-12T02:08:02.537 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-12 02:06:23.984384725 +0000 2024-03-12T02:08:02.537 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2024-03-12T02:08:02.537 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-12T02:08:02.592 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-12T02:08:02.592 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-12T02:08:02.592 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000426974 s, 1.2 MB/s 2024-03-12T02:08:02.593 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-12T02:08:02.642 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2024-03-12T02:08:02.692 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-12T02:08:02.692 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T02:08:02.692 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 508 Links: 1 2024-03-12T02:08:02.692 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T02:08:02.692 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-12 02:06:24.736367392 +0000 2024-03-12T02:08:02.693 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-12 02:06:24.476373386 +0000 2024-03-12T02:08:02.693 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-12 02:06:24.476373386 +0000 2024-03-12T02:08:02.693 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2024-03-12T02:08:02.693 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-12T02:08:02.746 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-12T02:08:02.746 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-12T02:08:02.746 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000370451 s, 1.4 MB/s 2024-03-12T02:08:02.747 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-12T02:08:02.798 DEBUG:teuthology.orchestra.run.smithi153:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-03-12T02:08:02.999 INFO:teuthology.orchestra.run.smithi153.stdout:loop 2024-03-12T02:08:03.001 INFO:tasks.nvme_loop:Connecting nvme_loop smithi153:/dev/vg_nvme/lv_1... 2024-03-12T02:08:03.001 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-03-12T02:08:03.027 INFO:teuthology.orchestra.run.smithi153.stdout:1 2024-03-12T02:08:03.054 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/vg_nvme/lv_11 2024-03-12T02:08:03.080 INFO:tasks.nvme_loop:Connecting nvme_loop smithi153:/dev/vg_nvme/lv_2... 2024-03-12T02:08:03.080 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-03-12T02:08:03.142 INFO:teuthology.orchestra.run.smithi153.stdout:1 2024-03-12T02:08:03.168 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/vg_nvme/lv_21 2024-03-12T02:08:03.192 INFO:tasks.nvme_loop:Connecting nvme_loop smithi153:/dev/vg_nvme/lv_3... 2024-03-12T02:08:03.192 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-03-12T02:08:03.255 INFO:teuthology.orchestra.run.smithi153.stdout:1 2024-03-12T02:08:03.277 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/vg_nvme/lv_31 2024-03-12T02:08:03.300 INFO:tasks.nvme_loop:Connecting nvme_loop smithi153:/dev/vg_nvme/lv_4... 2024-03-12T02:08:03.301 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-03-12T02:08:03.362 INFO:teuthology.orchestra.run.smithi153.stdout:1 2024-03-12T02:08:03.387 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/vg_nvme/lv_41 2024-03-12T02:08:03.409 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:08:03.409 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T02:08:03.460 DEBUG:teuthology.orchestra.run.smithi153:> sudo nvme list 2024-03-12T02:08:03.516 INFO:teuthology.orchestra.run.smithi153.stdout:Node SN Model Namespace Usage Format FW Rev 2024-03-12T02:08:03.516 INFO:teuthology.orchestra.run.smithi153.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-03-12T02:08:03.516 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/nvme0n1 CVFT623300MV400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV10171 2024-03-12T02:08:03.516 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/nvme1n1 66cad7e9715b5346e918 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-1 2024-03-12T02:08:03.516 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/nvme2n1 2d3ee23c267222953e9a Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-1 2024-03-12T02:08:03.516 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/nvme3n1 ff4d93fbcd80f7918942 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-1 2024-03-12T02:08:03.516 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/nvme4n1 09d4539be4bf518a66af Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-1 2024-03-12T02:08:03.517 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-03-12T02:08:03.517 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:08:03.517 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/scratch_devs 2024-03-12T02:08:03.579 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T02:08:03.579 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T02:08:03.586 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-12T02:08:03.587 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2024-03-12T02:08:03.637 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-12T02:08:03.637 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T02:08:03.638 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 492 Links: 1 2024-03-12T02:08:03.638 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T02:08:03.638 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-12 02:06:24.321684437 +0000 2024-03-12T02:08:03.638 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-12 02:06:24.073690156 +0000 2024-03-12T02:08:03.638 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-12 02:06:24.073690156 +0000 2024-03-12T02:08:03.638 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-03-12T02:08:03.638 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-12T02:08:03.693 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-12T02:08:03.693 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-12T02:08:03.693 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.00042767 s, 1.2 MB/s 2024-03-12T02:08:03.694 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-12T02:08:03.743 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2024-03-12T02:08:03.793 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-12T02:08:03.794 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T02:08:03.794 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 495 Links: 1 2024-03-12T02:08:03.794 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T02:08:03.794 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-12 02:06:24.541679364 +0000 2024-03-12T02:08:03.794 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-12 02:06:24.537679456 +0000 2024-03-12T02:08:03.794 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-12 02:06:24.537679456 +0000 2024-03-12T02:08:03.794 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-03-12T02:08:03.794 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-12T02:08:03.847 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-12T02:08:03.847 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-12T02:08:03.847 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000428416 s, 1.2 MB/s 2024-03-12T02:08:03.848 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-12T02:08:03.899 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2024-03-12T02:08:03.949 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-12T02:08:03.950 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T02:08:03.950 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 501 Links: 1 2024-03-12T02:08:03.950 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T02:08:03.950 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-12 02:06:25.289662114 +0000 2024-03-12T02:08:03.950 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-12 02:06:25.013668479 +0000 2024-03-12T02:08:03.950 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-12 02:06:25.013668479 +0000 2024-03-12T02:08:03.950 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-03-12T02:08:03.950 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-12T02:08:04.003 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-12T02:08:04.003 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-12T02:08:04.003 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000417749 s, 1.2 MB/s 2024-03-12T02:08:04.004 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-12T02:08:04.055 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2024-03-12T02:08:04.105 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-12T02:08:04.106 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-12T02:08:04.106 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 507 Links: 1 2024-03-12T02:08:04.106 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-12T02:08:04.106 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-12 02:06:25.501657225 +0000 2024-03-12T02:08:04.106 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-12 02:06:25.497657317 +0000 2024-03-12T02:08:04.106 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-12 02:06:25.497657317 +0000 2024-03-12T02:08:04.106 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-03-12T02:08:04.106 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-12T02:08:04.159 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-12T02:08:04.159 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-12T02:08:04.159 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000378389 s, 1.4 MB/s 2024-03-12T02:08:04.160 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-12T02:08:04.211 DEBUG:teuthology.orchestra.run.smithi163:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-03-12T02:08:04.343 INFO:teuthology.orchestra.run.smithi163.stdout:loop 2024-03-12T02:08:04.345 INFO:tasks.nvme_loop:Connecting nvme_loop smithi163:/dev/vg_nvme/lv_1... 2024-03-12T02:08:04.345 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-03-12T02:08:04.369 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-03-12T02:08:04.395 INFO:teuthology.orchestra.run.smithi163.stdout:/dev/vg_nvme/lv_11 2024-03-12T02:08:04.418 INFO:tasks.nvme_loop:Connecting nvme_loop smithi163:/dev/vg_nvme/lv_2... 2024-03-12T02:08:04.418 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-03-12T02:08:04.485 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-03-12T02:08:04.510 INFO:teuthology.orchestra.run.smithi163.stdout:/dev/vg_nvme/lv_21 2024-03-12T02:08:04.531 INFO:tasks.nvme_loop:Connecting nvme_loop smithi163:/dev/vg_nvme/lv_3... 2024-03-12T02:08:04.531 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-03-12T02:08:04.601 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-03-12T02:08:04.629 INFO:teuthology.orchestra.run.smithi163.stdout:/dev/vg_nvme/lv_31 2024-03-12T02:08:04.653 INFO:tasks.nvme_loop:Connecting nvme_loop smithi163:/dev/vg_nvme/lv_4... 2024-03-12T02:08:04.653 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-03-12T02:08:04.716 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-03-12T02:08:04.738 INFO:teuthology.orchestra.run.smithi163.stdout:/dev/vg_nvme/lv_41 2024-03-12T02:08:04.762 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T02:08:04.763 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T02:08:04.814 DEBUG:teuthology.orchestra.run.smithi163:> sudo nvme list 2024-03-12T02:08:04.872 INFO:teuthology.orchestra.run.smithi163.stdout:Node SN Model Namespace Usage Format FW Rev 2024-03-12T02:08:04.872 INFO:teuthology.orchestra.run.smithi163.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-03-12T02:08:04.872 INFO:teuthology.orchestra.run.smithi163.stdout:/dev/nvme0n1 CVFT623300GU400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-03-12T02:08:04.872 INFO:teuthology.orchestra.run.smithi163.stdout:/dev/nvme1n1 5fc5180e4fa8982c5e82 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-1 2024-03-12T02:08:04.872 INFO:teuthology.orchestra.run.smithi163.stdout:/dev/nvme2n1 ab937578e92804d56195 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-1 2024-03-12T02:08:04.872 INFO:teuthology.orchestra.run.smithi163.stdout:/dev/nvme3n1 5d5434f9542ba7a370ce Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-1 2024-03-12T02:08:04.872 INFO:teuthology.orchestra.run.smithi163.stdout:/dev/nvme4n1 550a77cac533c9459986 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.15.0-1 2024-03-12T02:08:04.873 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-03-12T02:08:04.873 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T02:08:04.874 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/scratch_devs 2024-03-12T02:08:04.929 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-12T02:08:05.001 INFO:tasks.cephadm:Config: {'roleless': True, 'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'c2ea977b6956d1f694ccf66acc692fdd979e3baa'} 2024-03-12T02:08:05.002 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa 2024-03-12T02:08:05.002 INFO:tasks.cephadm:Cluster fsid is 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:08:05.002 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-12T02:08:05.002 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-12T02:08:05.002 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi153': '172.21.15.153', 'mon.smithi163': '172.21.15.163'} 2024-03-12T02:08:05.003 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-12T02:08:05.003 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname $(hostname -s) 2024-03-12T02:08:05.018 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname $(hostname -s) 2024-03-12T02:08:05.035 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-12T02:08:05.035 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=c2ea977b6956d1f694ccf66acc692fdd979e3baa 2024-03-12T02:08:05.256 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c2ea977b6956d1f694ccf66acc692fdd979e3baa', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=focal,DIST=focal,MACHINE_SIZE=gigantic/54563/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.1-746-gc2ea977b', 'node_name': '172.21.2.15+braggi15', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=focal,DIST=focal,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.1-746-gc2ea977b-1focal'}, 'url': 'https://2.chacra.ceph.com/r/ceph/reef/c2ea977b6956d1f694ccf66acc692fdd979e3baa/ubuntu/focal/flavors/default/', 'distro_codename': 'focal', 'modified': '2024-03-09 16:32:54.164538', 'distro_version': '20.04', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/reef/c2ea977b6956d1f694ccf66acc692fdd979e3baa/ubuntu/focal/flavors/default/', 'archs': ['x86_64'], 'distro': 'ubuntu'}] 2024-03-12T02:08:05.470 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref reef, sha1 c2ea977b6956d1f694ccf66acc692fdd979e3baa from https://shaman.ceph.com/api/search/?project=ceph&distros=ubuntu%2F20.04%2Fx86_64&flavor=default&sha1=c2ea977b6956d1f694ccf66acc692fdd979e3baa 2024-03-12T02:08:05.471 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/reef/c2ea977b6956d1f694ccf66acc692fdd979e3baa/ubuntu/focal/x86_64/flavors/default/cephadm 2024-03-12T02:08:05.472 DEBUG:teuthology.orchestra.run.smithi153:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/reef/c2ea977b6956d1f694ccf66acc692fdd979e3baa/ubuntu/focal/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-12T02:08:06.066 INFO:teuthology.orchestra.run.smithi153.stdout:-rw-rw-r-- 1 ubuntu ubuntu 214548 Mar 12 02:08 /home/ubuntu/cephtest/cephadm 2024-03-12T02:08:06.067 DEBUG:teuthology.orchestra.run.smithi163:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/reef/c2ea977b6956d1f694ccf66acc692fdd979e3baa/ubuntu/focal/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-12T02:08:06.427 INFO:teuthology.orchestra.run.smithi163.stdout:-rw-rw-r-- 1 ubuntu ubuntu 214548 Mar 12 02:08 /home/ubuntu/cephtest/cephadm 2024-03-12T02:08:06.427 DEBUG:teuthology.orchestra.run.smithi153:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-12T02:08:06.436 DEBUG:teuthology.orchestra.run.smithi163:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-12T02:08:06.464 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa on all hosts... 2024-03-12T02:08:06.464 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa pull 2024-03-12T02:08:06.483 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa pull 2024-03-12T02:08:06.680 INFO:teuthology.orchestra.run.smithi153.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa... 2024-03-12T02:08:06.693 INFO:teuthology.orchestra.run.smithi163.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa... 2024-03-12T02:08:52.665 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2024-03-12T02:08:52.665 INFO:teuthology.orchestra.run.smithi163.stdout: "ceph_version": "ceph version 18.2.1-746-gc2ea977b (c2ea977b6956d1f694ccf66acc692fdd979e3baa) reef (stable)", 2024-03-12T02:08:52.665 INFO:teuthology.orchestra.run.smithi163.stdout: "image_id": "a30af69eda2283e161ea78b9ece83575a37489b95e40bb349921e5898d7f6746", 2024-03-12T02:08:52.665 INFO:teuthology.orchestra.run.smithi163.stdout: "repo_digests": [ 2024-03-12T02:08:52.666 INFO:teuthology.orchestra.run.smithi163.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:18efbf839983d010d4c22988cccbe24636b73671175cdbb8c34e60abb9a22633" 2024-03-12T02:08:52.666 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2024-03-12T02:08:52.666 INFO:teuthology.orchestra.run.smithi163.stdout:} 2024-03-12T02:08:52.689 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-12T02:08:52.689 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 18.2.1-746-gc2ea977b (c2ea977b6956d1f694ccf66acc692fdd979e3baa) reef (stable)", 2024-03-12T02:08:52.689 INFO:teuthology.orchestra.run.smithi153.stdout: "image_id": "a30af69eda2283e161ea78b9ece83575a37489b95e40bb349921e5898d7f6746", 2024-03-12T02:08:52.689 INFO:teuthology.orchestra.run.smithi153.stdout: "repo_digests": [ 2024-03-12T02:08:52.689 INFO:teuthology.orchestra.run.smithi153.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:18efbf839983d010d4c22988cccbe24636b73671175cdbb8c34e60abb9a22633" 2024-03-12T02:08:52.689 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-03-12T02:08:52.689 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-12T02:08:52.713 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph 2024-03-12T02:08:52.728 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph 2024-03-12T02:08:52.745 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /etc/ceph 2024-03-12T02:08:52.787 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /etc/ceph 2024-03-12T02:08:52.804 INFO:tasks.cephadm:Writing seed config... 2024-03-12T02:08:52.805 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-12T02:08:52.805 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-12T02:08:52.805 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-12T02:08:52.806 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-12T02:08:52.806 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-12T02:08:52.806 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-12T02:08:52.806 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-12T02:08:52.806 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-12T02:08:52.806 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:08:52.806 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-12T02:08:52.836 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5cbf0f4e-e015-11ee-95c8-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-03-12T02:08:52.836 DEBUG:teuthology.orchestra.run.smithi153:mon.smithi153> sudo journalctl -f -n 0 -u ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mon.smithi153.service 2024-03-12T02:08:52.879 INFO:tasks.cephadm:Bootstrapping... 2024-03-12T02:08:52.879 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa -v bootstrap --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.153 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-12T02:08:53.047 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-12T02:08:53.047 INFO:teuthology.orchestra.run.smithi153.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa', '-v', 'bootstrap', '--fsid', '5cbf0f4e-e015-11ee-95c8-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.153', '--skip-admin-label'] 2024-03-12T02:08:53.048 INFO:teuthology.orchestra.run.smithi153.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-12T02:08:53.048 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying podman|docker is present... 2024-03-12T02:08:53.048 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying lvm2 is present... 2024-03-12T02:08:53.048 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying time synchronization is in place... 2024-03-12T02:08:53.054 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-12T02:08:53.054 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-12T02:08:53.060 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-12T02:08:53.060 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-12T02:08:53.066 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2024-03-12T02:08:53.066 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2024-03-12T02:08:53.070 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2024-03-12T02:08:53.071 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-12T02:08:53.076 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2024-03-12T02:08:53.076 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout masked 2024-03-12T02:08:53.083 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2024-03-12T02:08:53.083 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-12T02:08:53.088 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2024-03-12T02:08:53.088 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2024-03-12T02:08:53.093 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2024-03-12T02:08:53.093 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-12T02:08:53.102 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout enabled 2024-03-12T02:08:53.109 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout active 2024-03-12T02:08:53.109 INFO:teuthology.orchestra.run.smithi153.stdout:Unit ntp.service is enabled and running 2024-03-12T02:08:53.109 INFO:teuthology.orchestra.run.smithi153.stdout:Repeating the final host check... 2024-03-12T02:08:53.109 INFO:teuthology.orchestra.run.smithi153.stdout:docker (/usr/bin/docker) is present 2024-03-12T02:08:53.109 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl is present 2024-03-12T02:08:53.109 INFO:teuthology.orchestra.run.smithi153.stdout:lvcreate is present 2024-03-12T02:08:53.114 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-12T02:08:53.115 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-12T02:08:53.120 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-12T02:08:53.120 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-12T02:08:53.126 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2024-03-12T02:08:53.126 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2024-03-12T02:08:53.131 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2024-03-12T02:08:53.131 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-12T02:08:53.137 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2024-03-12T02:08:53.137 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout masked 2024-03-12T02:08:53.144 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2024-03-12T02:08:53.144 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-12T02:08:53.149 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2024-03-12T02:08:53.149 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2024-03-12T02:08:53.156 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2024-03-12T02:08:53.156 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-12T02:08:53.162 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout enabled 2024-03-12T02:08:53.169 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-03-12T02:08:53.170 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout active 2024-03-12T02:08:53.171 INFO:teuthology.orchestra.run.smithi153.stdout:Unit ntp.service is enabled and running 2024-03-12T02:08:53.171 INFO:teuthology.orchestra.run.smithi153.stdout:Host looks OK 2024-03-12T02:08:53.171 INFO:teuthology.orchestra.run.smithi153.stdout:Cluster fsid: 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:08:53.171 INFO:teuthology.orchestra.run.smithi153.stdout:Acquiring lock 140557455158912 on /run/cephadm/5cbf0f4e-e015-11ee-95c8-87774f69a715.lock 2024-03-12T02:08:53.171 INFO:teuthology.orchestra.run.smithi153.stdout:Lock 140557455158912 acquired on /run/cephadm/5cbf0f4e-e015-11ee-95c8-87774f69a715.lock 2024-03-12T02:08:53.171 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying IP 172.21.15.153 port 3300 ... 2024-03-12T02:08:53.171 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying IP 172.21.15.153 port 6789 ... 2024-03-12T02:08:53.171 INFO:teuthology.orchestra.run.smithi153.stdout:Base mon IP(s) is [172.21.15.153:3300, 172.21.15.153:6789], mon addrv is [v2:172.21.15.153:3300,v1:172.21.15.153:6789] 2024-03-12T02:08:53.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 2024-03-12T02:08:53.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.153 metric 100 2024-03-12T02:08:53.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 2024-03-12T02:08:53.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.153 2024-03-12T02:08:53.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout 172.21.15.254 dev enp3s0f1 proto dhcp scope link src 172.21.15.153 metric 100 2024-03-12T02:08:53.179 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-12T02:08:53.179 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-12T02:08:53.179 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 expires 1783sec pref medium 2024-03-12T02:08:53.183 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-12T02:08:53.183 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-12T02:08:53.184 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-12T02:08:53.184 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-12T02:08:53.184 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:6deb/64 scope link 2024-03-12T02:08:53.184 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-12T02:08:53.184 INFO:teuthology.orchestra.run.smithi153.stdout:Mon IP `172.21.15.153` is in CIDR network `172.21.0.0/20` 2024-03-12T02:08:53.185 INFO:teuthology.orchestra.run.smithi153.stdout:Mon IP `172.21.15.153` is in CIDR network `172.21.0.0/20` 2024-03-12T02:08:53.185 INFO:teuthology.orchestra.run.smithi153.stdout:Mon IP `172.21.15.153` is in CIDR network `172.21.15.254/32` 2024-03-12T02:08:53.185 INFO:teuthology.orchestra.run.smithi153.stdout:Mon IP `172.21.15.153` is in CIDR network `172.21.15.254/32` 2024-03-12T02:08:53.185 INFO:teuthology.orchestra.run.smithi153.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20', '172.21.15.254/32', '172.21.15.254/32'] 2024-03-12T02:08:53.185 INFO:teuthology.orchestra.run.smithi153.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-12T02:08:53.186 INFO:teuthology.orchestra.run.smithi153.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa... 2024-03-12T02:08:53.464 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/docker: stdout c2ea977b6956d1f694ccf66acc692fdd979e3baa: Pulling from ceph-ci/ceph 2024-03-12T02:08:53.464 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/docker: stdout Digest: sha256:18efbf839983d010d4c22988cccbe24636b73671175cdbb8c34e60abb9a22633 2024-03-12T02:08:53.464 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/docker: stdout Status: Image is up to date for quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa 2024-03-12T02:08:53.464 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/docker: stdout quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa 2024-03-12T02:08:55.027 INFO:teuthology.orchestra.run.smithi153.stdout:ceph: stdout ceph version 18.2.1-746-gc2ea977b (c2ea977b6956d1f694ccf66acc692fdd979e3baa) reef (stable) 2024-03-12T02:08:55.028 INFO:teuthology.orchestra.run.smithi153.stdout:Ceph version: ceph version 18.2.1-746-gc2ea977b (c2ea977b6956d1f694ccf66acc692fdd979e3baa) reef (stable) 2024-03-12T02:08:55.028 INFO:teuthology.orchestra.run.smithi153.stdout:Extracting ceph user uid/gid from container image... 2024-03-12T02:08:56.398 INFO:teuthology.orchestra.run.smithi153.stdout:stat: stdout 167 167 2024-03-12T02:08:56.399 INFO:teuthology.orchestra.run.smithi153.stdout:Creating initial keys... 2024-03-12T02:08:57.855 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-authtool: stdout AQA5ue9lGf3RCxAA0649pjUEbpSaeZkf6p6S5w== 2024-03-12T02:08:59.302 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-authtool: stdout AQA6ue9lrXQbJxAAwe9bSrpjt1gD0cBwd7lteA== 2024-03-12T02:09:00.716 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-authtool: stdout AQA8ue9lcHzdBRAAaHGc4tDse51hDxu20K8P7w== 2024-03-12T02:09:00.718 INFO:teuthology.orchestra.run.smithi153.stdout:Creating initial monmap... 2024-03-12T02:09:02.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-12T02:09:02.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-12T02:09:02.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:09:02.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-12T02:09:02.130 INFO:teuthology.orchestra.run.smithi153.stdout:monmaptool for smithi153 [v2:172.21.15.153:3300,v1:172.21.15.153:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-12T02:09:02.130 INFO:teuthology.orchestra.run.smithi153.stdout:setting min_mon_release = pacific 2024-03-12T02:09:02.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: set fsid to 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:09:02.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-12T02:09:02.130 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:09:02.131 INFO:teuthology.orchestra.run.smithi153.stdout:Creating mon... 2024-03-12T02:09:03.517 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.933+0000 7f2185e0fc80 0 set uid:gid to 167:167 (ceph:ceph) 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.933+0000 7f2185e0fc80 1 imported monmap: 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr epoch 0 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr last_changed 2024-03-12T02:09:01.494884+0000 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr created 2024-03-12T02:09:01.494884+0000 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr min_mon_release 16 (pacific) 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr election_strategy: 1 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 0: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.smithi153 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.933+0000 7f2185e0fc80 0 /usr/bin/ceph-mon: set fsid to 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: RocksDB version: 7.9.2 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.518 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Git sha 0 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Compile date 2024-03-09 15:03:01 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: DB SUMMARY 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: DB Session ID: MTJ0WEKC3MWX1T7C2Z0Q 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi153/store.db dir, Total Num: 0, files: 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi153/store.db: 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.error_if_exists: 0 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.create_if_missing: 1 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.paranoid_checks: 1 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-12T02:09:03.519 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.env: 0x5623a70b0720 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.fs: PosixFileSystem 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.info_log: 0x5623a8a96600 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.max_file_opening_threads: 16 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.statistics: (nil) 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.use_fsync: 0 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.max_log_file_size: 0 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.keep_log_file_num: 1000 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.recycle_log_file_num: 0 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.allow_fallocate: 1 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.allow_mmap_reads: 0 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.allow_mmap_writes: 0 2024-03-12T02:09:03.520 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.use_direct_reads: 0 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.create_missing_column_families: 0 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.db_log_dir: 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.wal_dir: 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.advise_random_on_open: 1 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.db_write_buffer_size: 0 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.write_buffer_manager: 0x5623a8ad7180 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-12T02:09:03.521 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.rate_limiter: (nil) 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.wal_recovery_mode: 2 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.enable_thread_tracking: 0 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.enable_pipelined_write: 0 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.unordered_write: 0 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.row_cache: None 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.wal_filter: None 2024-03-12T02:09:03.522 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.allow_ingest_behind: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.two_write_queues: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.manual_wal_flush: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.wal_compression: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.atomic_flush: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.log_readahead_size: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.best_efforts_recovery: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.allow_data_in_errors: 0 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.db_host_id: __hostname__ 2024-03-12T02:09:03.523 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.max_background_jobs: 2 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.max_background_compactions: -1 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.max_subcompactions: 1 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.max_total_wal_size: 0 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.max_open_files: -1 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.bytes_per_sync: 0 2024-03-12T02:09:03.524 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.compaction_readahead_size: 0 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Options.max_background_flushes: -1 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Compression algorithms supported: 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: kZSTD supported: 0 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: kXpressCompression supported: 0 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: kLZ4HCCompression supported: 1 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: kZlibCompression supported: 1 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: kSnappyCompression supported: 1 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: kLZ4Compression supported: 1 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: kBZip2Compression supported: 0 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.937+0000 7f2185e0fc80 4 rocksdb: [db/db_impl/db_impl_open.cc:317] Creating manifest 1 2024-03-12T02:09:03.525 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi153/store.db/MANIFEST-000001 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.merge_operator: 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_filter: None 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_filter_factory: None 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.sst_partitioner_factory: None 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5623a983e740) 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks: 1 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks_with_high_priority: 0 2024-03-12T02:09:03.526 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr pin_top_level_index_and_filter: 1 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr index_type: 0 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr data_block_index_type: 0 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr index_shortening: 1 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr data_block_hash_table_util_ratio: 0.750000 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr checksum: 4 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr no_block_cache: 0 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr block_cache: 0x5623a8b571f0 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr block_cache_name: BinnedLRUCache 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr block_cache_options: 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr capacity : 536870912 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr num_shard_bits : 4 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr strict_capacity_limit : 0 2024-03-12T02:09:03.527 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr high_pri_pool_ratio: 0.000 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr block_cache_compressed: (nil) 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr persistent_cache: (nil) 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr block_size: 4096 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr block_size_deviation: 10 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr block_restart_interval: 16 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr index_block_restart_interval: 1 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr metadata_block_size: 4096 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr partition_filters: 0 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr use_delta_encoding: 1 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr filter_policy: bloomfilter 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr whole_key_filtering: 1 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr verify_compression: 0 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr read_amp_bytes_per_bit: 0 2024-03-12T02:09:03.528 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr format_version: 5 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr enable_index_compression: 1 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr block_align: 0 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr max_auto_readahead_size: 262144 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr prepopulate_block_cache: 0 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr initial_auto_readahead_size: 8192 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr num_file_reads_for_auto_readahead: 2 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.write_buffer_size: 33554432 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_write_buffer_number: 2 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compression: NoCompression 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.bottommost_compression: Disabled 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.prefix_extractor: nullptr 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-12T02:09:03.529 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.num_levels: 7 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compression_opts.level: 32767 2024-03-12T02:09:03.530 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compression_opts.strategy: 0 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compression_opts.enabled: false 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.target_file_size_base: 67108864 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-12T02:09:03.531 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.arena_block_size: 1048576 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-12T02:09:03.532 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.disable_auto_compactions: 0 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.table_properties_collectors: 2024-03-12T02:09:03.533 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.inplace_update_support: 0 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.bloom_locality: 0 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.max_successive_merges: 0 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.paranoid_file_checks: 0 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.force_consistency_checks: 1 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.report_bg_io_stats: 0 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.ttl: 2592000 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-12T02:09:03.534 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.enable_blob_files: false 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.min_blob_size: 0 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.blob_file_size: 268435456 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.blob_file_starting_level: 0 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi153/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-12T02:09:03.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 69106362-f782-4948-a8bd-ee8e2558b641 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.941+0000 7f2185e0fc80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 5 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.945+0000 7f2185e0fc80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5623a8bce000 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.945+0000 7f2185e0fc80 4 rocksdb: DB pointer 0x5623a8bc0000 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.945+0000 7f217600f700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.945+0000 7f217600f700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr ** DB Stats ** 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T02:09:03.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T02:09:03.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Flush(GB): cumulative 0.000, interval 0.000 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr AddFile(GB): cumulative 0.000, interval 0.000 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr AddFile(Total Files): cumulative 0, interval 0 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr AddFile(L0 Files): cumulative 0, interval 0 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr AddFile(Keys): cumulative 0, interval 0 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Block cache BinnedLRUCache@0x5623a8b571f0#7 capacity: 512.00 MB usage: 0.00 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr Block cache entry stats(count,size,portion): Misc(1,0.00 KB,0%) 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr ** File Read Latency Histogram By Level [default] ** 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.945+0000 7f2185e0fc80 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.945+0000 7f2185e0fc80 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2024-03-12T02:09:03.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-12T02:09:02.945+0000 7f2185e0fc80 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-smithi153 for mon.smithi153 2024-03-12T02:09:03.539 INFO:teuthology.orchestra.run.smithi153.stdout:create mon.smithi153 on 2024-03-12T02:09:03.937 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2024-03-12T02:09:04.177 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715.target -> /etc/systemd/system/ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715.target. 2024-03-12T02:09:04.177 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715.target -> /etc/systemd/system/ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715.target. 2024-03-12T02:09:04.483 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mon.smithi153 2024-03-12T02:09:04.483 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mon.smithi153.service: Unit ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mon.smithi153.service not loaded. 2024-03-12T02:09:04.695 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715.target.wants/ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mon.smithi153.service -> /etc/systemd/system/ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@.service. 2024-03-12T02:09:04.705 INFO:teuthology.orchestra.run.smithi153.stdout:firewalld does not appear to be present 2024-03-12T02:09:04.705 INFO:teuthology.orchestra.run.smithi153.stdout:Not possible to enable service . firewalld.service is not available 2024-03-12T02:09:04.706 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mon to start... 2024-03-12T02:09:04.706 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mon... 2024-03-12T02:09:06.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:06 smithi153 bash[13829]: cluster 2024-03-12T02:09:06.081916+0000 mon.smithi153 (mon.0) 1 : cluster 1 mon.smithi153 is new leader, mons smithi153 in quorum (ranks 0) 2024-03-12T02:09:06.948 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout cluster: 2024-03-12T02:09:06.948 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout id: 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:09:06.948 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-12T02:09:06.948 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:06.948 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout services: 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi153 (age 0.183818s) 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout data: 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout pgs: 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:mon is available 2024-03-12T02:09:06.949 INFO:teuthology.orchestra.run.smithi153.stdout:Assimilating anything we can from ceph.conf... 2024-03-12T02:09:07.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:07 smithi153 bash[13829]: cluster 2024-03-12T02:09:06.106121+0000 mon.smithi153 (mon.0) 2 : cluster 1 mon.smithi153 is new leader, mons smithi153 in quorum (ranks 0) 2024-03-12T02:09:07.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:07 smithi153 bash[13829]: cluster 2024-03-12T02:09:06.106497+0000 mon.smithi153 (mon.0) 3 : cluster 0 monmap e1: 1 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T02:09:07.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:07 smithi153 bash[13829]: cluster 2024-03-12T02:09:06.111819+0000 mon.smithi153 (mon.0) 4 : cluster 0 fsmap 2024-03-12T02:09:07.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:07 smithi153 bash[13829]: cluster 2024-03-12T02:09:06.115117+0000 mon.smithi153 (mon.0) 5 : cluster 0 osdmap e1: 0 total, 0 up, 0 in 2024-03-12T02:09:07.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:07 smithi153 bash[13829]: cluster 2024-03-12T02:09:06.115571+0000 mon.smithi153 (mon.0) 6 : cluster 0 mgrmap e1: no daemons active 2024-03-12T02:09:07.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:07 smithi153 bash[13829]: audit 2024-03-12T02:09:06.289876+0000 mon.smithi153 (mon.0) 7 : audit 0 from='client.? 172.21.15.153:0/4094537863' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-12T02:09:08.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:08 smithi153 bash[13829]: audit 2024-03-12T02:09:07.962303+0000 mon.smithi153 (mon.0) 8 : audit 1 from='client.? 172.21.15.153:0/2409799742' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-12T02:09:08.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:08 smithi153 bash[13829]: audit 2024-03-12T02:09:07.968289+0000 mon.smithi153 (mon.0) 9 : audit 1 from='client.? 172.21.15.153:0/2409799742' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-12T02:09:08.636 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:08.636 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [global] 2024-03-12T02:09:08.636 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout fsid = 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:09:08.636 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.153:3300,v1:172.21.15.153:6789] 2024-03-12T02:09:08.636 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-12T02:09:08.636 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-12T02:09:08.636 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-12T02:09:08.636 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-12T02:09:08.636 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:08.636 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-12T02:09:08.636 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-12T02:09:08.637 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:08.637 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [osd] 2024-03-12T02:09:08.637 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-12T02:09:08.637 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-12T02:09:08.637 INFO:teuthology.orchestra.run.smithi153.stdout:Generating new minimal ceph.conf... 2024-03-12T02:09:10.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:09 smithi153 bash[13829]: audit 2024-03-12T02:09:09.708017+0000 mon.smithi153 (mon.0) 10 : audit 0 from='client.? 172.21.15.153:0/3995225498' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:09:10.358 INFO:teuthology.orchestra.run.smithi153.stdout:Restarting the monitor... 2024-03-12T02:09:10.669 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:10 smithi153 systemd[1]: Stopping Ceph mon.smithi153 for 5cbf0f4e-e015-11ee-95c8-87774f69a715... 2024-03-12T02:09:10.669 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:10 smithi153 bash[13829]: debug 2024-03-12T02:09:10.525+0000 7ffb96ce0700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.smithi153 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-03-12T02:09:10.669 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:10 smithi153 bash[13829]: debug 2024-03-12T02:09:10.525+0000 7ffb96ce0700 -1 mon.smithi153@0(leader) e1 *** Got Signal Terminated *** 2024-03-12T02:09:11.244 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:11 smithi153 bash[14241]: ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715-mon-smithi153 2024-03-12T02:09:11.266 INFO:teuthology.orchestra.run.smithi153.stdout:Setting public_network to 172.21.15.254/32,172.21.0.0/20 in mon config section 2024-03-12T02:09:11.523 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:11 smithi153 bash[14301]: Error response from daemon: No such container: ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715-mon-smithi153 2024-03-12T02:09:11.523 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:11 smithi153 systemd[1]: ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mon.smithi153.service: Succeeded. 2024-03-12T02:09:11.523 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:11 smithi153 systemd[1]: Stopped Ceph mon.smithi153 for 5cbf0f4e-e015-11ee-95c8-87774f69a715. 2024-03-12T02:09:11.524 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:11 smithi153 systemd[1]: Started Ceph mon.smithi153 for 5cbf0f4e-e015-11ee-95c8-87774f69a715. 2024-03-12T02:09:12.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.477+0000 7f41b4544c80 0 set uid:gid to 167:167 (ceph:ceph) 2024-03-12T02:09:12.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.477+0000 7f41b4544c80 0 ceph version 18.2.1-746-gc2ea977b (c2ea977b6956d1f694ccf66acc692fdd979e3baa) reef (stable), process ceph-mon, pid 7 2024-03-12T02:09:12.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.477+0000 7f41b4544c80 0 pidfile_write: ignore empty --pid-file 2024-03-12T02:09:12.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 0 load: jerasure load: lrc 2024-03-12T02:09:12.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: RocksDB version: 7.9.2 2024-03-12T02:09:12.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Git sha 0 2024-03-12T02:09:12.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Compile date 2024-03-09 15:03:01 2024-03-12T02:09:12.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: DB SUMMARY 2024-03-12T02:09:12.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: DB Session ID: D2L5HIM7S03A6B7LXE5L 2024-03-12T02:09:12.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: CURRENT file: CURRENT 2024-03-12T02:09:12.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: IDENTITY file: IDENTITY 2024-03-12T02:09:12.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-12T02:09:12.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi153/store.db dir, Total Num: 1, files: 000008.sst 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi153/store.db: 000009.log size: 83838 ; 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.error_if_exists: 0 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.create_if_missing: 0 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.paranoid_checks: 1 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.env: 0x55b928b6a720 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.fs: PosixFileSystem 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.info_log: 0x55b92ad453c0 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_file_opening_threads: 16 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.statistics: (nil) 2024-03-12T02:09:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.use_fsync: 0 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_log_file_size: 0 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.keep_log_file_num: 1000 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.recycle_log_file_num: 0 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.allow_fallocate: 1 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.allow_mmap_reads: 0 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.allow_mmap_writes: 0 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.use_direct_reads: 0 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.create_missing_column_families: 0 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.db_log_dir: 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.wal_dir: 2024-03-12T02:09:12.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.advise_random_on_open: 1 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.db_write_buffer_size: 0 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.write_buffer_manager: 0x55b929fda320 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.rate_limiter: (nil) 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-12T02:09:12.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.wal_recovery_mode: 2 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.enable_thread_tracking: 0 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.enable_pipelined_write: 0 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.unordered_write: 0 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.row_cache: None 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.wal_filter: None 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.allow_ingest_behind: 0 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.two_write_queues: 0 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.manual_wal_flush: 0 2024-03-12T02:09:12.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.wal_compression: 0 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.atomic_flush: 0 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.log_readahead_size: 0 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.best_efforts_recovery: 0 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.allow_data_in_errors: 0 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.db_host_id: __hostname__ 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_background_jobs: 2 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_background_compactions: -1 2024-03-12T02:09:12.827 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_subcompactions: 1 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_total_wal_size: 0 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_open_files: -1 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bytes_per_sync: 0 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_readahead_size: 0 2024-03-12T02:09:12.828 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_background_flushes: -1 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Compression algorithms supported: 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: kZSTD supported: 0 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: kXpressCompression supported: 0 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: kLZ4HCCompression supported: 1 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: kZlibCompression supported: 1 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: kSnappyCompression supported: 1 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: kLZ4Compression supported: 1 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: kBZip2Compression supported: 0 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-03-12T02:09:12.829 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi153/store.db/MANIFEST-000010 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.merge_operator: 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_filter: None 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_filter_factory: None 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.sst_partitioner_factory: None 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b92ad454e0) 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: cache_index_and_filter_blocks: 1 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: pin_top_level_index_and_filter: 1 2024-03-12T02:09:12.830 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: index_type: 0 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: data_block_index_type: 0 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: index_shortening: 1 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: data_block_hash_table_util_ratio: 0.750000 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: checksum: 4 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: no_block_cache: 0 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: block_cache: 0x55b92a05d1f0 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: block_cache_name: BinnedLRUCache 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: block_cache_options: 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: capacity : 536870912 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: num_shard_bits : 4 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: strict_capacity_limit : 0 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: high_pri_pool_ratio: 0.000 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: block_cache_compressed: (nil) 2024-03-12T02:09:12.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: persistent_cache: (nil) 2024-03-12T02:09:12.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: block_size: 4096 2024-03-12T02:09:12.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: block_size_deviation: 10 2024-03-12T02:09:12.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: block_restart_interval: 16 2024-03-12T02:09:12.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: index_block_restart_interval: 1 2024-03-12T02:09:12.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: metadata_block_size: 4096 2024-03-12T02:09:12.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: partition_filters: 0 2024-03-12T02:09:12.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: use_delta_encoding: 1 2024-03-12T02:09:12.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: filter_policy: bloomfilter 2024-03-12T02:09:12.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: whole_key_filtering: 1 2024-03-12T02:09:12.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: verify_compression: 0 2024-03-12T02:09:12.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: read_amp_bytes_per_bit: 0 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: format_version: 5 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: enable_index_compression: 1 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: block_align: 0 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: max_auto_readahead_size: 262144 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: prepopulate_block_cache: 0 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: initial_auto_readahead_size: 8192 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: num_file_reads_for_auto_readahead: 2 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.write_buffer_size: 33554432 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_write_buffer_number: 2 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compression: NoCompression 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bottommost_compression: Disabled 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.prefix_extractor: nullptr 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-12T02:09:12.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.num_levels: 7 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-03-12T02:09:12.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compression_opts.level: 32767 2024-03-12T02:09:12.835 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compression_opts.strategy: 0 2024-03-12T02:09:12.835 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-12T02:09:12.835 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-12T02:09:12.835 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-12T02:09:12.835 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-12T02:09:12.835 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compression_opts.enabled: false 2024-03-12T02:09:12.835 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.target_file_size_base: 67108864 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-12T02:09:12.836 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.arena_block_size: 1048576 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.disable_auto_compactions: 0 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-12T02:09:12.837 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.table_properties_collectors: 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.inplace_update_support: 0 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-03-12T02:09:12.838 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.bloom_locality: 0 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.max_successive_merges: 0 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.paranoid_file_checks: 0 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.force_consistency_checks: 1 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.report_bg_io_stats: 0 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.ttl: 2592000 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.enable_blob_files: false 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.min_blob_size: 0 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.blob_file_size: 268435456 2024-03-12T02:09:12.839 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.blob_file_starting_level: 0 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.481+0000 7f41b4544c80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.485+0000 7f41b4544c80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi153/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.485+0000 7f41b4544c80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.485+0000 7f41b4544c80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 69106362-f782-4948-a8bd-ee8e2558b641 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.485+0000 7f41b4544c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1710209352489364, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.485+0000 7f41b4544c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.485+0000 7f41b4544c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1710209352491028, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 80610, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 248, "table_properties": {"data_size": 78793, "index_size": 227, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9753, "raw_average_key_size": 47, "raw_value_size": 73213, "raw_average_value_size": 353, "num_data_blocks": 10, "num_entries": 207, "num_filter_entries": 207, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710209352, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "69106362-f782-4948-a8bd-ee8e2558b641", "db_session_id": "D2L5HIM7S03A6B7LXE5L", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.485+0000 7f41b4544c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1710209352491149, "job": 1, "event": "recovery_finished"} 2024-03-12T02:09:12.840 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.485+0000 7f41b4544c80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.485+0000 7f41b4544c80 4 rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi153/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.485+0000 7f41b4544c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b92a0fe000 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.485+0000 7f41b4544c80 4 rocksdb: DB pointer 0x55b92a0ec000 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.489+0000 7f41a2d3e700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.489+0000 7f41a2d3e700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: ** DB Stats ** 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T02:09:12.841 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: ** Compaction Stats [default] ** 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: L0 2/0 80.59 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Sum 2/0 80.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: ** Compaction Stats [default] ** 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 78.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Flush(GB): cumulative 0.000, interval 0.000 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-12T02:09:12.842 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: AddFile(Total Files): cumulative 0, interval 0 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: AddFile(L0 Files): cumulative 0, interval 0 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: AddFile(Keys): cumulative 0, interval 0 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Cumulative compaction: 0.00 GB write, 15.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Interval compaction: 0.00 GB write, 15.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Block cache BinnedLRUCache@0x55b92a05d1f0#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: ** File Read Latency Histogram By Level [default] ** 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.489+0000 7f41b4544c80 0 starting mon.smithi153 rank 0 at public addrs [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] at bind addrs [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi153 fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.489+0000 7f41b4544c80 1 mon.smithi153@-1(???) e1 preinit fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.489+0000 7f41b4544c80 0 mon.smithi153@-1(???).mds e1 new map 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.489+0000 7f41b4544c80 0 mon.smithi153@-1(???).mds e1 print_map 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: e1 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: enable_multiple, ever_enabled_multiple: 1,1 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-12T02:09:12.843 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: legacy client fscid: -1 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: No filesystems configured 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.489+0000 7f41b4544c80 0 mon.smithi153@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.489+0000 7f41b4544c80 0 mon.smithi153@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.489+0000 7f41b4544c80 0 mon.smithi153@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.489+0000 7f41b4544c80 0 mon.smithi153@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: debug 2024-03-12T02:09:12.489+0000 7f41b4544c80 1 mon.smithi153@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: cluster 2024-03-12T02:09:12.518063+0000 mon.smithi153 (mon.0) 1 : cluster 1 mon.smithi153 is new leader, mons smithi153 in quorum (ranks 0) 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: cluster 2024-03-12T02:09:12.518406+0000 mon.smithi153 (mon.0) 2 : cluster 0 monmap e1: 1 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: cluster 2024-03-12T02:09:12.538091+0000 mon.smithi153 (mon.0) 3 : cluster 0 fsmap 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: cluster 2024-03-12T02:09:12.538156+0000 mon.smithi153 (mon.0) 4 : cluster 0 osdmap e1: 0 total, 0 up, 0 in 2024-03-12T02:09:12.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:12 smithi153 bash[14340]: cluster 2024-03-12T02:09:12.538836+0000 mon.smithi153 (mon.0) 5 : cluster 0 mgrmap e1: no daemons active 2024-03-12T02:09:13.441 INFO:teuthology.orchestra.run.smithi153.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-12T02:09:13.467 INFO:teuthology.orchestra.run.smithi153.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-12T02:09:13.467 INFO:teuthology.orchestra.run.smithi153.stdout:Creating mgr... 2024-03-12T02:09:13.468 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-12T02:09:13.468 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-12T02:09:13.468 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying port 0.0.0.0:8443 ... 2024-03-12T02:09:13.777 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mgr.smithi153.asomxr 2024-03-12T02:09:13.777 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mgr.smithi153.asomxr.service: Unit ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mgr.smithi153.asomxr.service not loaded. 2024-03-12T02:09:13.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:13 smithi153 bash[14340]: audit 2024-03-12T02:09:12.795819+0000 mon.smithi153 (mon.0) 6 : audit 1 from='client.? 172.21.15.153:0/3914597878' entity='client.admin' 2024-03-12T02:09:13.985 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715.target.wants/ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mgr.smithi153.asomxr.service -> /etc/systemd/system/ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@.service. 2024-03-12T02:09:13.997 INFO:teuthology.orchestra.run.smithi153.stdout:firewalld does not appear to be present 2024-03-12T02:09:13.997 INFO:teuthology.orchestra.run.smithi153.stdout:Not possible to enable service . firewalld.service is not available 2024-03-12T02:09:13.997 INFO:teuthology.orchestra.run.smithi153.stdout:firewalld does not appear to be present 2024-03-12T02:09:13.997 INFO:teuthology.orchestra.run.smithi153.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-03-12T02:09:13.997 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mgr to start... 2024-03-12T02:09:13.997 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mgr... 2024-03-12T02:09:15.749 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:15 smithi153 bash[14340]: audit 2024-03-12T02:09:15.516200+0000 mon.smithi153 (mon.0) 7 : audit 0 from='client.? 172.21.15.153:0/3318505790' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-12T02:09:16.237 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:16.237 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-12T02:09:16.237 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsid": "5cbf0f4e-e015-11ee-95c8-87774f69a715", 2024-03-12T02:09:16.237 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "health": { 2024-03-12T02:09:16.237 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-12T02:09:16.237 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-12T02:09:16.237 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-12T02:09:16.237 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 0 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "smithi153" 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:16.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-12T02:09:16.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:16.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-12T02:09:16.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-12T02:09:16.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-12T02:09:16.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-12T02:09:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-12T02:09:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-12T02:09:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-12T02:09:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-12T02:09:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "restful" 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modified": "2024-03-12T02:09:06.109379+0000", 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-12T02:09:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:mgr not available, waiting (1/15)... 2024-03-12T02:09:19.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:19 smithi153 bash[14340]: audit 2024-03-12T02:09:19.422452+0000 mon.smithi153 (mon.0) 8 : audit 0 from='client.? 172.21.15.153:0/1753479564' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-12T02:09:20.158 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:20.158 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-12T02:09:20.158 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsid": "5cbf0f4e-e015-11ee-95c8-87774f69a715", 2024-03-12T02:09:20.158 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "health": { 2024-03-12T02:09:20.158 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-12T02:09:20.158 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-12T02:09:20.158 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-12T02:09:20.158 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 0 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "smithi153" 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:20.159 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-12T02:09:20.160 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:20.160 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-12T02:09:20.160 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-12T02:09:20.160 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-12T02:09:20.160 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-12T02:09:20.160 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-12T02:09:20.160 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-12T02:09:20.160 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:20.163 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-12T02:09:20.163 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-12T02:09:20.163 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-12T02:09:20.163 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-12T02:09:20.163 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-12T02:09:20.163 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-12T02:09:20.164 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "restful" 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modified": "2024-03-12T02:09:06.109379+0000", 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-12T02:09:20.165 INFO:teuthology.orchestra.run.smithi153.stdout:mgr not available, waiting (2/15)... 2024-03-12T02:09:23.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:23 smithi153 bash[14340]: audit 2024-03-12T02:09:23.496769+0000 mon.smithi153 (mon.0) 9 : audit 0 from='client.? 172.21.15.153:0/76882004' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-12T02:09:24.058 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:24.058 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-12T02:09:24.058 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsid": "5cbf0f4e-e015-11ee-95c8-87774f69a715", 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "health": { 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 0 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "smithi153" 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-12T02:09:24.059 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-12T02:09:24.060 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-12T02:09:24.061 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-12T02:09:24.062 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-12T02:09:24.062 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-12T02:09:24.062 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-12T02:09:24.062 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-12T02:09:24.062 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "restful" 2024-03-12T02:09:24.062 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:24.062 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T02:09:24.065 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:24.065 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-12T02:09:24.065 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:24.065 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modified": "2024-03-12T02:09:06.109379+0000", 2024-03-12T02:09:24.066 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T02:09:24.066 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:24.066 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-12T02:09:24.066 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-12T02:09:24.066 INFO:teuthology.orchestra.run.smithi153.stdout:mgr not available, waiting (3/15)... 2024-03-12T02:09:24.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: cluster 2024-03-12T02:09:23.923994+0000 mon.smithi153 (mon.0) 10 : cluster 1 Activating manager daemon smithi153.asomxr 2024-03-12T02:09:24.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: cluster 2024-03-12T02:09:23.927158+0000 mon.smithi153 (mon.0) 11 : cluster 0 mgrmap e2: smithi153.asomxr(active, starting, since 0.00366936s) 2024-03-12T02:09:24.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: audit 2024-03-12T02:09:23.930631+0000 mon.smithi153 (mon.0) 12 : audit 0 from='mgr.14100 172.21.15.153:0/2008092976' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T02:09:24.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: audit 2024-03-12T02:09:23.930903+0000 mon.smithi153 (mon.0) 13 : audit 0 from='mgr.14100 172.21.15.153:0/2008092976' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T02:09:24.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: audit 2024-03-12T02:09:23.931284+0000 mon.smithi153 (mon.0) 14 : audit 0 from='mgr.14100 172.21.15.153:0/2008092976' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T02:09:24.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: audit 2024-03-12T02:09:23.931687+0000 mon.smithi153 (mon.0) 15 : audit 0 from='mgr.14100 172.21.15.153:0/2008092976' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-12T02:09:24.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: audit 2024-03-12T02:09:23.931984+0000 mon.smithi153 (mon.0) 16 : audit 0 from='mgr.14100 172.21.15.153:0/2008092976' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr metadata", "who": "smithi153.asomxr", "id": "smithi153.asomxr"}]: dispatch 2024-03-12T02:09:24.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: cluster 2024-03-12T02:09:23.956698+0000 mon.smithi153 (mon.0) 17 : cluster 1 Manager daemon smithi153.asomxr is now available 2024-03-12T02:09:24.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: audit 2024-03-12T02:09:23.975823+0000 mon.smithi153 (mon.0) 18 : audit 1 from='mgr.14100 172.21.15.153:0/2008092976' entity='mgr.smithi153.asomxr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.asomxr/mirror_snapshot_schedule"}]: dispatch 2024-03-12T02:09:24.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: audit 2024-03-12T02:09:23.980659+0000 mon.smithi153 (mon.0) 19 : audit 1 from='mgr.14100 172.21.15.153:0/2008092976' entity='mgr.smithi153.asomxr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.asomxr/trash_purge_schedule"}]: dispatch 2024-03-12T02:09:24.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: audit 2024-03-12T02:09:23.984960+0000 mon.smithi153 (mon.0) 20 : audit 1 from='mgr.14100 172.21.15.153:0/2008092976' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:24.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: audit 2024-03-12T02:09:23.987846+0000 mon.smithi153 (mon.0) 21 : audit 1 from='mgr.14100 172.21.15.153:0/2008092976' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:24.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:24 smithi153 bash[14340]: audit 2024-03-12T02:09:23.990471+0000 mon.smithi153 (mon.0) 22 : audit 1 from='mgr.14100 172.21.15.153:0/2008092976' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:26.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:25 smithi153 bash[14340]: cluster 2024-03-12T02:09:24.936751+0000 mon.smithi153 (mon.0) 23 : cluster 0 mgrmap e3: smithi153.asomxr(active, since 1.01325s) 2024-03-12T02:09:28.248 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:28.248 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-12T02:09:28.248 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsid": "5cbf0f4e-e015-11ee-95c8-87774f69a715", 2024-03-12T02:09:28.248 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "health": { 2024-03-12T02:09:28.248 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-12T02:09:28.248 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-12T02:09:28.248 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-12T02:09:28.248 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:28.248 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-12T02:09:28.248 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 0 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "smithi153" 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-12T02:09:28.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-12T02:09:28.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "restful" 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-12T02:09:28.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T02:09:28.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:28.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-12T02:09:28.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-12T02:09:28.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modified": "2024-03-12T02:09:06.109379+0000", 2024-03-12T02:09:28.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-12T02:09:28.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-12T02:09:28.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-12T02:09:28.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-12T02:09:28.252 INFO:teuthology.orchestra.run.smithi153.stdout:mgr is available 2024-03-12T02:09:28.275 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:27 smithi153 bash[14340]: cluster 2024-03-12T02:09:26.933744+0000 mon.smithi153 (mon.0) 24 : cluster 0 mgrmap e4: smithi153.asomxr(active, since 3s) 2024-03-12T02:09:28.275 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:27 smithi153 bash[14340]: audit 2024-03-12T02:09:27.602191+0000 mon.smithi153 (mon.0) 25 : audit 0 from='client.? 172.21.15.153:0/3967294586' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-12T02:09:30.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:29 smithi153 bash[14340]: audit 2024-03-12T02:09:29.723088+0000 mon.smithi153 (mon.0) 26 : audit 1 from='client.? 172.21.15.153:0/1828850050' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-12T02:09:30.363 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:30.363 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [global] 2024-03-12T02:09:30.363 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout fsid = 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:09:30.363 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-12T02:09:30.364 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-12T02:09:30.364 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-12T02:09:30.364 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-12T02:09:30.364 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:30.364 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-12T02:09:30.364 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-12T02:09:30.364 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-12T02:09:30.364 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [osd] 2024-03-12T02:09:30.364 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-12T02:09:30.364 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-12T02:09:30.364 INFO:teuthology.orchestra.run.smithi153.stdout:Enabling cephadm module... 2024-03-12T02:09:32.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:31 smithi153 bash[14340]: audit 2024-03-12T02:09:31.852478+0000 mon.smithi153 (mon.0) 27 : audit 1 from='client.? 172.21.15.153:0/3897238604' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-12T02:09:33.196 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:32 smithi153 bash[14340]: audit 2024-03-12T02:09:31.909597+0000 mon.smithi153 (mon.0) 28 : audit 1 from='client.? 172.21.15.153:0/3897238604' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-12T02:09:33.197 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:32 smithi153 bash[14340]: cluster 2024-03-12T02:09:31.909703+0000 mon.smithi153 (mon.0) 29 : cluster 0 mgrmap e5: smithi153.asomxr(active, since 7s) 2024-03-12T02:09:34.288 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:33 smithi153 bash[14340]: audit 2024-03-12T02:09:33.832402+0000 mon.smithi153 (mon.0) 30 : audit 0 from='client.? 172.21.15.153:0/4265232637' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-12T02:09:34.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-12T02:09:34.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-12T02:09:34.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-12T02:09:34.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "active_name": "smithi153.asomxr", 2024-03-12T02:09:34.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-12T02:09:34.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-12T02:09:34.538 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for the mgr to restart... 2024-03-12T02:09:34.538 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mgr epoch 5... 2024-03-12T02:09:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:40 smithi153 bash[14340]: cluster 2024-03-12T02:09:40.347400+0000 mon.smithi153 (mon.0) 31 : cluster 1 Active manager daemon smithi153.asomxr restarted 2024-03-12T02:09:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:40 smithi153 bash[14340]: cluster 2024-03-12T02:09:40.348026+0000 mon.smithi153 (mon.0) 32 : cluster 1 Activating manager daemon smithi153.asomxr 2024-03-12T02:09:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:40 smithi153 bash[14340]: cluster 2024-03-12T02:09:40.356012+0000 mon.smithi153 (mon.0) 33 : cluster 0 osdmap e2: 0 total, 0 up, 0 in 2024-03-12T02:09:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:40 smithi153 bash[14340]: cluster 2024-03-12T02:09:40.356159+0000 mon.smithi153 (mon.0) 34 : cluster 0 mgrmap e6: smithi153.asomxr(active, starting, since 0.00832206s) 2024-03-12T02:09:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:40 smithi153 bash[14340]: audit 2024-03-12T02:09:40.358038+0000 mon.smithi153 (mon.0) 35 : audit 0 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-12T02:09:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:40 smithi153 bash[14340]: audit 2024-03-12T02:09:40.359042+0000 mon.smithi153 (mon.0) 36 : audit 0 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr metadata", "who": "smithi153.asomxr", "id": "smithi153.asomxr"}]: dispatch 2024-03-12T02:09:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:40 smithi153 bash[14340]: audit 2024-03-12T02:09:40.359812+0000 mon.smithi153 (mon.0) 37 : audit 0 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T02:09:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:40 smithi153 bash[14340]: audit 2024-03-12T02:09:40.360047+0000 mon.smithi153 (mon.0) 38 : audit 0 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T02:09:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:40 smithi153 bash[14340]: audit 2024-03-12T02:09:40.360256+0000 mon.smithi153 (mon.0) 39 : audit 0 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T02:09:40.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:40 smithi153 bash[14340]: cluster 2024-03-12T02:09:40.384801+0000 mon.smithi153 (mon.0) 40 : cluster 1 Manager daemon smithi153.asomxr is now available 2024-03-12T02:09:41.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:41 smithi153 bash[14340]: cephadm 2024-03-12T02:09:40.405207+0000 mgr.smithi153.asomxr (mgr.14120) 1 : cephadm 1 Found migration_current of "None". Setting to last migration. 2024-03-12T02:09:41.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:41 smithi153 bash[14340]: audit 2024-03-12T02:09:40.411700+0000 mon.smithi153 (mon.0) 41 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:41.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:41 smithi153 bash[14340]: audit 2024-03-12T02:09:40.418905+0000 mon.smithi153 (mon.0) 42 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:41.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:41 smithi153 bash[14340]: audit 2024-03-12T02:09:40.426466+0000 mon.smithi153 (mon.0) 43 : audit 0 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:09:41.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:41 smithi153 bash[14340]: audit 2024-03-12T02:09:40.430892+0000 mon.smithi153 (mon.0) 44 : audit 0 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:09:41.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:41 smithi153 bash[14340]: audit 2024-03-12T02:09:40.435098+0000 mon.smithi153 (mon.0) 45 : audit 0 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:09:41.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:41 smithi153 bash[14340]: audit 2024-03-12T02:09:40.445901+0000 mon.smithi153 (mon.0) 46 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.asomxr/mirror_snapshot_schedule"}]: dispatch 2024-03-12T02:09:41.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:41 smithi153 bash[14340]: audit 2024-03-12T02:09:40.450885+0000 mon.smithi153 (mon.0) 47 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.asomxr/trash_purge_schedule"}]: dispatch 2024-03-12T02:09:41.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:41 smithi153 bash[14340]: audit 2024-03-12T02:09:40.659561+0000 mon.smithi153 (mon.0) 48 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:41.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:41 smithi153 bash[14340]: audit 2024-03-12T02:09:40.663595+0000 mon.smithi153 (mon.0) 49 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:41.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:41 smithi153 bash[14340]: cluster 2024-03-12T02:09:41.358055+0000 mon.smithi153 (mon.0) 50 : cluster 0 mgrmap e7: smithi153.asomxr(active, since 1.01022s) 2024-03-12T02:09:42.043 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-12T02:09:42.043 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-12T02:09:42.043 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-12T02:09:42.043 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-12T02:09:42.044 INFO:teuthology.orchestra.run.smithi153.stdout:mgr epoch 5 is available 2024-03-12T02:09:42.044 INFO:teuthology.orchestra.run.smithi153.stdout:Setting orchestrator backend to cephadm... 2024-03-12T02:09:42.738 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:42 smithi153 bash[14340]: cephadm 2024-03-12T02:09:41.302315+0000 mgr.smithi153.asomxr (mgr.14120) 2 : cephadm 1 [12/Mar/2024:02:09:41] ENGINE Bus STARTING 2024-03-12T02:09:42.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:42 smithi153 bash[14340]: audit 2024-03-12T02:09:41.358284+0000 mgr.smithi153.asomxr (mgr.14120) 3 : audit 0 from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-12T02:09:42.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:42 smithi153 bash[14340]: audit 2024-03-12T02:09:41.374407+0000 mgr.smithi153.asomxr (mgr.14120) 4 : audit 0 from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-12T02:09:42.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:42 smithi153 bash[14340]: cephadm 2024-03-12T02:09:41.404614+0000 mgr.smithi153.asomxr (mgr.14120) 5 : cephadm 1 [12/Mar/2024:02:09:41] ENGINE Serving on http://172.21.15.153:8765 2024-03-12T02:09:42.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:42 smithi153 bash[14340]: cephadm 2024-03-12T02:09:41.527655+0000 mgr.smithi153.asomxr (mgr.14120) 6 : cephadm 1 [12/Mar/2024:02:09:41] ENGINE Serving on https://172.21.15.153:7150 2024-03-12T02:09:42.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:42 smithi153 bash[14340]: cephadm 2024-03-12T02:09:41.528073+0000 mgr.smithi153.asomxr (mgr.14120) 7 : cephadm 1 [12/Mar/2024:02:09:41] ENGINE Bus STARTED 2024-03-12T02:09:42.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:42 smithi153 bash[14340]: audit 2024-03-12T02:09:41.558732+0000 mon.smithi153 (mon.0) 51 : audit 0 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:09:43.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:43 smithi153 bash[14340]: cluster 2024-03-12T02:09:42.418128+0000 mon.smithi153 (mon.0) 52 : cluster 0 mgrmap e8: smithi153.asomxr(active, since 2s) 2024-03-12T02:09:43.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:43 smithi153 bash[14340]: audit 2024-03-12T02:09:43.411869+0000 mon.smithi153 (mon.0) 53 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:44.763 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:44 smithi153 bash[14340]: audit 2024-03-12T02:09:43.402915+0000 mgr.smithi153.asomxr (mgr.14120) 8 : audit 0 from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:09:44.763 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:44 smithi153 bash[14340]: audit 2024-03-12T02:09:43.430558+0000 mon.smithi153 (mon.0) 54 : audit 0 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:09:46.064 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-12T02:09:46.065 INFO:teuthology.orchestra.run.smithi153.stdout:Generating ssh key... 2024-03-12T02:09:46.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:46 smithi153 bash[14340]: audit 2024-03-12T02:09:45.407593+0000 mgr.smithi153.asomxr (mgr.14120) 9 : audit 0 from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:09:49.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:48 smithi153 bash[14340]: audit 2024-03-12T02:09:47.477355+0000 mgr.smithi153.asomxr (mgr.14120) 10 : audit 0 from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:09:49.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:48 smithi153 bash[14340]: cephadm 2024-03-12T02:09:47.477580+0000 mgr.smithi153.asomxr (mgr.14120) 11 : cephadm 1 Generating ssh key... 2024-03-12T02:09:49.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:48 smithi153 bash[14340]: audit 2024-03-12T02:09:47.875406+0000 mon.smithi153 (mon.0) 55 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:49.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:48 smithi153 bash[14340]: audit 2024-03-12T02:09:47.878934+0000 mon.smithi153 (mon.0) 56 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:50.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9izVK64HuL9Fub3Sc29OIVpZq29ovNIBd/ZzZlbJs/8s6bW/hg0YtTMzO01Jp/kcuUJRXhvENjxz7JUFQj9TxpIUUxlelbo6C2gMCnrhlQq/H9myT9lAgYy7Uj3K830C+AwimZOTuKzRoiXoBm1ZLOnnw/ZveyejfP+9/RociDsMjMK4RjXA4LdLfvUUT+s/FoO+H6guYnIrPCrqxIlmw3F69QGAXxX3Ry3yV+7JQB5hmJFDcNUdRB3dcqefJ+1cFL6nEkM+DL6UhUsukjb7IfxEIIhkYOLVeuNEaMabNlTClGq6VUZW+SJJxCl8h7/3sYaPObkozMV/UcICA5pgJOaeDAR85bRGPKLjDPzEtvze1sozaAA6lBeTGWQ3IK1TlAyze0k3f36/3DTq43mdumvOO8IpFgaqO3f+rQFICj/k2P81BFm21PvmCdRbryXZeEw64yLiNj0MjLm8MgoFvvftccaGhcEqiXHizhvW0fygK7irH+SFtll7vjKlV21U= ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:09:50.628 INFO:teuthology.orchestra.run.smithi153.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-12T02:09:50.628 INFO:teuthology.orchestra.run.smithi153.stdout:Adding key to root@localhost authorized_keys... 2024-03-12T02:09:50.628 INFO:teuthology.orchestra.run.smithi153.stdout:Adding host smithi153... 2024-03-12T02:09:51.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:51 smithi153 bash[14340]: audit 2024-03-12T02:09:49.970344+0000 mgr.smithi153.asomxr (mgr.14120) 12 : audit 0 from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:09:53.542 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:53 smithi153 bash[14340]: audit 2024-03-12T02:09:52.029129+0000 mgr.smithi153.asomxr (mgr.14120) 13 : audit 0 from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "addr": "172.21.15.153", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:09:53.542 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:53 smithi153 bash[14340]: cephadm 2024-03-12T02:09:52.806331+0000 mgr.smithi153.asomxr (mgr.14120) 14 : cephadm 1 Deploying cephadm binary to smithi153 2024-03-12T02:09:56.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Added host 'smithi153' with addr '172.21.15.153' 2024-03-12T02:09:56.685 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying mon service with default placement... 2024-03-12T02:09:57.088 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:56 smithi153 bash[14340]: audit 2024-03-12T02:09:55.607097+0000 mon.smithi153 (mon.0) 57 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:57.089 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:56 smithi153 bash[14340]: cephadm 2024-03-12T02:09:55.607883+0000 mgr.smithi153.asomxr (mgr.14120) 15 : cephadm 1 Added host smithi153 2024-03-12T02:09:57.089 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:56 smithi153 bash[14340]: audit 2024-03-12T02:09:55.626417+0000 mon.smithi153 (mon.0) 58 : audit 0 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:09:59.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-12T02:09:59.086 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying mgr service with default placement... 2024-03-12T02:09:59.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:59 smithi153 bash[14340]: audit 2024-03-12T02:09:58.453936+0000 mgr.smithi153.asomxr (mgr.14120) 16 : audit 0 from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:09:59.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:59 smithi153 bash[14340]: cephadm 2024-03-12T02:09:58.455213+0000 mgr.smithi153.asomxr (mgr.14120) 17 : cephadm 1 Saving service mon spec with placement count:5 2024-03-12T02:09:59.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:59 smithi153 bash[14340]: audit 2024-03-12T02:09:58.459883+0000 mon.smithi153 (mon.0) 59 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:59.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:59 smithi153 bash[14340]: audit 2024-03-12T02:09:58.509525+0000 mon.smithi153 (mon.0) 60 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:09:59.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:09:59 smithi153 bash[14340]: audit 2024-03-12T02:09:58.907992+0000 mon.smithi153 (mon.0) 61 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:01.179 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-12T02:10:01.179 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying crash service with default placement... 2024-03-12T02:10:01.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:01 smithi153 bash[14340]: cluster 2024-03-12T02:10:00.361653+0000 mgr.smithi153.asomxr (mgr.14120) 18 : cluster 0 pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:01.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:01 smithi153 bash[14340]: audit 2024-03-12T02:10:00.489172+0000 mgr.smithi153.asomxr (mgr.14120) 19 : audit 0 from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:10:01.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:01 smithi153 bash[14340]: cephadm 2024-03-12T02:10:00.490112+0000 mgr.smithi153.asomxr (mgr.14120) 20 : cephadm 1 Saving service mgr spec with placement count:2 2024-03-12T02:10:01.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:01 smithi153 bash[14340]: audit 2024-03-12T02:10:00.494506+0000 mon.smithi153 (mon.0) 62 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:03.318 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-12T02:10:03.318 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying ceph-exporter service with default placement... 2024-03-12T02:10:03.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:03 smithi153 bash[14340]: cluster 2024-03-12T02:10:02.362124+0000 mgr.smithi153.asomxr (mgr.14120) 21 : cluster 0 pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:03 smithi153 bash[14340]: audit 2024-03-12T02:10:02.573241+0000 mgr.smithi153.asomxr (mgr.14120) 22 : audit 0 from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:10:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:03 smithi153 bash[14340]: cephadm 2024-03-12T02:10:02.575264+0000 mgr.smithi153.asomxr (mgr.14120) 23 : cephadm 1 Saving service crash spec with placement * 2024-03-12T02:10:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:03 smithi153 bash[14340]: audit 2024-03-12T02:10:02.579973+0000 mon.smithi153 (mon.0) 63 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:05.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:05 smithi153 bash[14340]: audit 2024-03-12T02:10:04.091950+0000 mon.smithi153 (mon.0) 64 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:05.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:05 smithi153 bash[14340]: cluster 2024-03-12T02:10:04.362435+0000 mgr.smithi153.asomxr (mgr.14120) 24 : cluster 0 pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:05.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:05 smithi153 bash[14340]: audit 2024-03-12T02:10:04.774033+0000 mgr.smithi153.asomxr (mgr.14120) 25 : audit 0 from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:10:05.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:05 smithi153 bash[14340]: cephadm 2024-03-12T02:10:04.775556+0000 mgr.smithi153.asomxr (mgr.14120) 26 : cephadm 1 Saving service ceph-exporter spec with placement * 2024-03-12T02:10:05.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:05 smithi153 bash[14340]: audit 2024-03-12T02:10:04.779959+0000 mon.smithi153 (mon.0) 65 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:05.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:05 smithi153 bash[14340]: audit 2024-03-12T02:10:04.884632+0000 mon.smithi153 (mon.0) 66 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:05.440 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-03-12T02:10:05.440 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying prometheus service with default placement... 2024-03-12T02:10:07.596 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-12T02:10:07.596 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying grafana service with default placement... 2024-03-12T02:10:08.316 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:07 smithi153 bash[14340]: cluster 2024-03-12T02:10:06.362752+0000 mgr.smithi153.asomxr (mgr.14120) 27 : cluster 0 pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:08.317 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:07 smithi153 bash[14340]: audit 2024-03-12T02:10:06.901838+0000 mgr.smithi153.asomxr (mgr.14120) 28 : audit 0 from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:10:08.317 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:07 smithi153 bash[14340]: cephadm 2024-03-12T02:10:06.904110+0000 mgr.smithi153.asomxr (mgr.14120) 29 : cephadm 1 Saving service prometheus spec with placement count:1 2024-03-12T02:10:08.317 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:07 smithi153 bash[14340]: audit 2024-03-12T02:10:06.911780+0000 mon.smithi153 (mon.0) 67 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:09.615 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-12T02:10:09.615 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying node-exporter service with default placement... 2024-03-12T02:10:10.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:09 smithi153 bash[14340]: cluster 2024-03-12T02:10:08.363091+0000 mgr.smithi153.asomxr (mgr.14120) 30 : cluster 0 pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:10.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:09 smithi153 bash[14340]: audit 2024-03-12T02:10:08.964605+0000 mgr.smithi153.asomxr (mgr.14120) 31 : audit 0 from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:10:10.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:09 smithi153 bash[14340]: cephadm 2024-03-12T02:10:08.966927+0000 mgr.smithi153.asomxr (mgr.14120) 32 : cephadm 1 Saving service grafana spec with placement count:1 2024-03-12T02:10:10.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:09 smithi153 bash[14340]: audit 2024-03-12T02:10:08.973355+0000 mon.smithi153 (mon.0) 68 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:12.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:11 smithi153 bash[14340]: cluster 2024-03-12T02:10:10.363479+0000 mgr.smithi153.asomxr (mgr.14120) 33 : cluster 0 pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:12.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:11 smithi153 bash[14340]: audit 2024-03-12T02:10:11.562084+0000 mon.smithi153 (mon.0) 69 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:12.504 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-12T02:10:12.504 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying alertmanager service with default placement... 2024-03-12T02:10:13.224 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:12 smithi153 bash[14340]: audit 2024-03-12T02:10:11.557157+0000 mgr.smithi153.asomxr (mgr.14120) 34 : audit 0 from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:10:13.224 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:12 smithi153 bash[14340]: cephadm 2024-03-12T02:10:11.558864+0000 mgr.smithi153.asomxr (mgr.14120) 35 : cephadm 1 Saving service node-exporter spec with placement * 2024-03-12T02:10:14.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:13 smithi153 bash[14340]: cluster 2024-03-12T02:10:12.363936+0000 mgr.smithi153.asomxr (mgr.14120) 36 : cluster 0 pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:14.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:13 smithi153 bash[14340]: audit 2024-03-12T02:10:13.894257+0000 mon.smithi153 (mon.0) 70 : audit 1 from='mgr.14120 172.21.15.153:0/191857140' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:14.595 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-12T02:10:15.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:14 smithi153 bash[14340]: audit 2024-03-12T02:10:13.888280+0000 mgr.smithi153.asomxr (mgr.14120) 37 : audit 0 from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:10:15.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:14 smithi153 bash[14340]: cephadm 2024-03-12T02:10:13.890524+0000 mgr.smithi153.asomxr (mgr.14120) 38 : cephadm 1 Saving service alertmanager spec with placement count:1 2024-03-12T02:10:16.215 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:15 smithi153 bash[14340]: cluster 2024-03-12T02:10:14.364399+0000 mgr.smithi153.asomxr (mgr.14120) 39 : cluster 0 pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:17.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:16 smithi153 bash[14340]: audit 2024-03-12T02:10:15.943445+0000 mon.smithi153 (mon.0) 71 : audit 1 from='client.? 172.21.15.153:0/2640635358' entity='client.admin' 2024-03-12T02:10:18.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:17 smithi153 bash[14340]: cluster 2024-03-12T02:10:16.364826+0000 mgr.smithi153.asomxr (mgr.14120) 40 : cluster 0 pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:18.811 INFO:teuthology.orchestra.run.smithi153.stdout:Enabling the dashboard module... 2024-03-12T02:10:19.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:19 smithi153 bash[14340]: audit 2024-03-12T02:10:18.069929+0000 mon.smithi153 (mon.0) 72 : audit 1 from='client.? 172.21.15.153:0/1646876206' entity='client.admin' 2024-03-12T02:10:19.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:19 smithi153 bash[14340]: cluster 2024-03-12T02:10:18.365248+0000 mgr.smithi153.asomxr (mgr.14120) 41 : cluster 0 pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:20.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:20 smithi153 bash[14340]: audit 2024-03-12T02:10:20.279941+0000 mon.smithi153 (mon.0) 73 : audit 1 from='client.? 172.21.15.153:0/3346391027' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-12T02:10:21.637 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:21 smithi153 bash[14340]: audit 2024-03-12T02:10:20.337578+0000 mon.smithi153 (mon.0) 74 : audit 1 from='client.? 172.21.15.153:0/3346391027' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-12T02:10:21.637 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:21 smithi153 bash[14340]: cluster 2024-03-12T02:10:20.337740+0000 mon.smithi153 (mon.0) 75 : cluster 0 mgrmap e9: smithi153.asomxr(active, since 39s) 2024-03-12T02:10:23.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-12T02:10:23.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-12T02:10:23.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-12T02:10:23.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "active_name": "smithi153.asomxr", 2024-03-12T02:10:23.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-12T02:10:23.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-12T02:10:23.253 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for the mgr to restart... 2024-03-12T02:10:23.253 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mgr epoch 9... 2024-03-12T02:10:23.570 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:23 smithi153 bash[14340]: audit 2024-03-12T02:10:22.535904+0000 mon.smithi153 (mon.0) 76 : audit 0 from='client.? 172.21.15.153:0/3990897060' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-12T02:10:29.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:29 smithi153 bash[14340]: cluster 2024-03-12T02:10:29.011389+0000 mon.smithi153 (mon.0) 77 : cluster 1 Active manager daemon smithi153.asomxr restarted 2024-03-12T02:10:29.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:29 smithi153 bash[14340]: cluster 2024-03-12T02:10:29.011882+0000 mon.smithi153 (mon.0) 78 : cluster 1 Activating manager daemon smithi153.asomxr 2024-03-12T02:10:29.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:29 smithi153 bash[14340]: cluster 2024-03-12T02:10:29.020331+0000 mon.smithi153 (mon.0) 79 : cluster 0 osdmap e3: 0 total, 0 up, 0 in 2024-03-12T02:10:29.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:29 smithi153 bash[14340]: cluster 2024-03-12T02:10:29.020535+0000 mon.smithi153 (mon.0) 80 : cluster 0 mgrmap e10: smithi153.asomxr(active, starting, since 0.00886087s) 2024-03-12T02:10:29.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:29 smithi153 bash[14340]: audit 2024-03-12T02:10:29.021478+0000 mon.smithi153 (mon.0) 81 : audit 0 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-12T02:10:29.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:29 smithi153 bash[14340]: audit 2024-03-12T02:10:29.022815+0000 mon.smithi153 (mon.0) 82 : audit 0 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr metadata", "who": "smithi153.asomxr", "id": "smithi153.asomxr"}]: dispatch 2024-03-12T02:10:29.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:29 smithi153 bash[14340]: audit 2024-03-12T02:10:29.023587+0000 mon.smithi153 (mon.0) 83 : audit 0 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T02:10:29.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:29 smithi153 bash[14340]: audit 2024-03-12T02:10:29.023819+0000 mon.smithi153 (mon.0) 84 : audit 0 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T02:10:29.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:29 smithi153 bash[14340]: audit 2024-03-12T02:10:29.024029+0000 mon.smithi153 (mon.0) 85 : audit 0 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T02:10:29.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:29 smithi153 bash[14340]: cluster 2024-03-12T02:10:29.053878+0000 mon.smithi153 (mon.0) 86 : cluster 1 Manager daemon smithi153.asomxr is now available 2024-03-12T02:10:30.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:30 smithi153 bash[14340]: audit 2024-03-12T02:10:29.085390+0000 mon.smithi153 (mon.0) 87 : audit 0 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:10:30.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:30 smithi153 bash[14340]: audit 2024-03-12T02:10:29.088781+0000 mon.smithi153 (mon.0) 88 : audit 0 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:10:30.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:30 smithi153 bash[14340]: audit 2024-03-12T02:10:29.114671+0000 mon.smithi153 (mon.0) 89 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.asomxr/mirror_snapshot_schedule"}]: dispatch 2024-03-12T02:10:30.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:30 smithi153 bash[14340]: audit 2024-03-12T02:10:29.119072+0000 mon.smithi153 (mon.0) 90 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.asomxr/trash_purge_schedule"}]: dispatch 2024-03-12T02:10:30.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:30 smithi153 bash[14340]: cluster 2024-03-12T02:10:30.023304+0000 mon.smithi153 (mon.0) 91 : cluster 0 mgrmap e11: smithi153.asomxr(active, since 1.01162s) 2024-03-12T02:10:30.776 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-12T02:10:30.776 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-12T02:10:30.776 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-12T02:10:30.776 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-12T02:10:30.776 INFO:teuthology.orchestra.run.smithi153.stdout:mgr epoch 9 is available 2024-03-12T02:10:30.776 INFO:teuthology.orchestra.run.smithi153.stdout:Generating a dashboard self-signed certificate... 2024-03-12T02:10:31.515 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:31 smithi153 bash[14340]: cephadm 2024-03-12T02:10:29.657323+0000 mgr.smithi153.asomxr (mgr.14164) 1 : cephadm 1 [12/Mar/2024:02:10:29] ENGINE Bus STARTING 2024-03-12T02:10:31.515 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:31 smithi153 bash[14340]: cephadm 2024-03-12T02:10:29.759540+0000 mgr.smithi153.asomxr (mgr.14164) 2 : cephadm 1 [12/Mar/2024:02:10:29] ENGINE Serving on http://172.21.15.153:8765 2024-03-12T02:10:31.515 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:31 smithi153 bash[14340]: cephadm 2024-03-12T02:10:29.887537+0000 mgr.smithi153.asomxr (mgr.14164) 3 : cephadm 1 [12/Mar/2024:02:10:29] ENGINE Serving on https://172.21.15.153:7150 2024-03-12T02:10:31.515 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:31 smithi153 bash[14340]: cephadm 2024-03-12T02:10:29.887868+0000 mgr.smithi153.asomxr (mgr.14164) 4 : cephadm 1 [12/Mar/2024:02:10:29] ENGINE Bus STARTED 2024-03-12T02:10:31.516 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:31 smithi153 bash[14340]: audit 2024-03-12T02:10:30.022416+0000 mgr.smithi153.asomxr (mgr.14164) 5 : audit 0 from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-12T02:10:31.516 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:31 smithi153 bash[14340]: audit 2024-03-12T02:10:30.035570+0000 mgr.smithi153.asomxr (mgr.14164) 6 : audit 0 from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-12T02:10:32.345 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:32 smithi153 bash[14340]: cluster 2024-03-12T02:10:31.072037+0000 mon.smithi153 (mon.0) 92 : cluster 0 mgrmap e12: smithi153.asomxr(active, since 2s) 2024-03-12T02:10:32.972 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-12T02:10:32.972 INFO:teuthology.orchestra.run.smithi153.stdout:Creating initial admin user... 2024-03-12T02:10:33.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:33 smithi153 bash[14340]: audit 2024-03-12T02:10:32.110862+0000 mgr.smithi153.asomxr (mgr.14164) 7 : audit 0 from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:10:33.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:33 smithi153 bash[14340]: audit 2024-03-12T02:10:32.257002+0000 mon.smithi153 (mon.0) 93 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:33.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:33 smithi153 bash[14340]: audit 2024-03-12T02:10:32.261827+0000 mon.smithi153 (mon.0) 94 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:35.371 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$UQd8x81.MgKGDfpTifaEPehthrUTK3u09h.uGXr26kRLdTdYdkS1m", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710209434, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-12T02:10:35.371 INFO:teuthology.orchestra.run.smithi153.stdout:Fetching dashboard port number... 2024-03-12T02:10:36.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:35 smithi153 bash[14340]: audit 2024-03-12T02:10:34.434204+0000 mgr.smithi153.asomxr (mgr.14164) 8 : audit 0 from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:10:36.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:35 smithi153 bash[14340]: audit 2024-03-12T02:10:34.675798+0000 mon.smithi153 (mon.0) 95 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:37.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:36 smithi153 bash[14340]: cluster 2024-03-12T02:10:35.680649+0000 mon.smithi153 (mon.0) 96 : cluster 0 mgrmap e13: smithi153.asomxr(active, since 6s) 2024-03-12T02:10:37.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:36 smithi153 bash[14340]: audit 2024-03-12T02:10:36.101728+0000 mon.smithi153 (mon.0) 97 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:37.502 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 8443 2024-03-12T02:10:37.502 INFO:teuthology.orchestra.run.smithi153.stdout:firewalld does not appear to be present 2024-03-12T02:10:37.502 INFO:teuthology.orchestra.run.smithi153.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-12T02:10:37.505 INFO:teuthology.orchestra.run.smithi153.stdout:Ceph Dashboard is now available at: 2024-03-12T02:10:37.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:10:37.505 INFO:teuthology.orchestra.run.smithi153.stdout: URL: https://smithi153.front.sepia.ceph.com:8443/ 2024-03-12T02:10:37.505 INFO:teuthology.orchestra.run.smithi153.stdout: User: admin 2024-03-12T02:10:37.505 INFO:teuthology.orchestra.run.smithi153.stdout: Password: t2euyow8md 2024-03-12T02:10:37.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:10:37.506 INFO:teuthology.orchestra.run.smithi153.stdout:Saving cluster configuration to /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config directory 2024-03-12T02:10:37.506 INFO:teuthology.orchestra.run.smithi153.stdout:Enabling autotune for osd_memory_target 2024-03-12T02:10:37.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:37 smithi153 bash[14340]: audit 2024-03-12T02:10:36.815348+0000 mon.smithi153 (mon.0) 98 : audit 0 from='client.? 172.21.15.153:0/322674333' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-12T02:10:37.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:37 smithi153 bash[14340]: audit 2024-03-12T02:10:36.939599+0000 mon.smithi153 (mon.0) 99 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:41.929 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout:Or, if you are only running a single cluster on this host: 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout: ceph telemetry on 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:10:41.930 INFO:teuthology.orchestra.run.smithi153.stdout:For more information see: 2024-03-12T02:10:41.931 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:10:41.931 INFO:teuthology.orchestra.run.smithi153.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-12T02:10:41.931 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:10:41.931 INFO:teuthology.orchestra.run.smithi153.stdout:Bootstrap complete. 2024-03-12T02:10:41.972 INFO:tasks.cephadm:Fetching config... 2024-03-12T02:10:41.973 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:10:41.973 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-12T02:10:41.979 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-12T02:10:41.979 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:10:41.979 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-12T02:10:42.027 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-12T02:10:42.027 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:10:42.027 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/keyring of=/dev/stdout 2024-03-12T02:10:42.079 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-12T02:10:42.079 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:10:42.079 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-12T02:10:42.126 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-12T02:10:42.127 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9izVK64HuL9Fub3Sc29OIVpZq29ovNIBd/ZzZlbJs/8s6bW/hg0YtTMzO01Jp/kcuUJRXhvENjxz7JUFQj9TxpIUUxlelbo6C2gMCnrhlQq/H9myT9lAgYy7Uj3K830C+AwimZOTuKzRoiXoBm1ZLOnnw/ZveyejfP+9/RociDsMjMK4RjXA4LdLfvUUT+s/FoO+H6guYnIrPCrqxIlmw3F69QGAXxX3Ry3yV+7JQB5hmJFDcNUdRB3dcqefJ+1cFL6nEkM+DL6UhUsukjb7IfxEIIhkYOLVeuNEaMabNlTClGq6VUZW+SJJxCl8h7/3sYaPObkozMV/UcICA5pgJOaeDAR85bRGPKLjDPzEtvze1sozaAA6lBeTGWQ3IK1TlAyze0k3f36/3DTq43mdumvOO8IpFgaqO3f+rQFICj/k2P81BFm21PvmCdRbryXZeEw64yLiNj0MjLm8MgoFvvftccaGhcEqiXHizhvW0fygK7irH+SFtll7vjKlV21U= ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-12T02:10:42.192 INFO:teuthology.orchestra.run.smithi153.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9izVK64HuL9Fub3Sc29OIVpZq29ovNIBd/ZzZlbJs/8s6bW/hg0YtTMzO01Jp/kcuUJRXhvENjxz7JUFQj9TxpIUUxlelbo6C2gMCnrhlQq/H9myT9lAgYy7Uj3K830C+AwimZOTuKzRoiXoBm1ZLOnnw/ZveyejfP+9/RociDsMjMK4RjXA4LdLfvUUT+s/FoO+H6guYnIrPCrqxIlmw3F69QGAXxX3Ry3yV+7JQB5hmJFDcNUdRB3dcqefJ+1cFL6nEkM+DL6UhUsukjb7IfxEIIhkYOLVeuNEaMabNlTClGq6VUZW+SJJxCl8h7/3sYaPObkozMV/UcICA5pgJOaeDAR85bRGPKLjDPzEtvze1sozaAA6lBeTGWQ3IK1TlAyze0k3f36/3DTq43mdumvOO8IpFgaqO3f+rQFICj/k2P81BFm21PvmCdRbryXZeEw64yLiNj0MjLm8MgoFvvftccaGhcEqiXHizhvW0fygK7irH+SFtll7vjKlV21U= ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:10:42.199 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9izVK64HuL9Fub3Sc29OIVpZq29ovNIBd/ZzZlbJs/8s6bW/hg0YtTMzO01Jp/kcuUJRXhvENjxz7JUFQj9TxpIUUxlelbo6C2gMCnrhlQq/H9myT9lAgYy7Uj3K830C+AwimZOTuKzRoiXoBm1ZLOnnw/ZveyejfP+9/RociDsMjMK4RjXA4LdLfvUUT+s/FoO+H6guYnIrPCrqxIlmw3F69QGAXxX3Ry3yV+7JQB5hmJFDcNUdRB3dcqefJ+1cFL6nEkM+DL6UhUsukjb7IfxEIIhkYOLVeuNEaMabNlTClGq6VUZW+SJJxCl8h7/3sYaPObkozMV/UcICA5pgJOaeDAR85bRGPKLjDPzEtvze1sozaAA6lBeTGWQ3IK1TlAyze0k3f36/3DTq43mdumvOO8IpFgaqO3f+rQFICj/k2P81BFm21PvmCdRbryXZeEw64yLiNj0MjLm8MgoFvvftccaGhcEqiXHizhvW0fygK7irH+SFtll7vjKlV21U= ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-12T02:10:42.222 INFO:teuthology.orchestra.run.smithi163.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9izVK64HuL9Fub3Sc29OIVpZq29ovNIBd/ZzZlbJs/8s6bW/hg0YtTMzO01Jp/kcuUJRXhvENjxz7JUFQj9TxpIUUxlelbo6C2gMCnrhlQq/H9myT9lAgYy7Uj3K830C+AwimZOTuKzRoiXoBm1ZLOnnw/ZveyejfP+9/RociDsMjMK4RjXA4LdLfvUUT+s/FoO+H6guYnIrPCrqxIlmw3F69QGAXxX3Ry3yV+7JQB5hmJFDcNUdRB3dcqefJ+1cFL6nEkM+DL6UhUsukjb7IfxEIIhkYOLVeuNEaMabNlTClGq6VUZW+SJJxCl8h7/3sYaPObkozMV/UcICA5pgJOaeDAR85bRGPKLjDPzEtvze1sozaAA6lBeTGWQ3IK1TlAyze0k3f36/3DTq43mdumvOO8IpFgaqO3f+rQFICj/k2P81BFm21PvmCdRbryXZeEw64yLiNj0MjLm8MgoFvvftccaGhcEqiXHizhvW0fygK7irH+SFtll7vjKlV21U= ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:10:42.231 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-12T02:10:42.257 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:42 smithi153 bash[14340]: audit 2024-03-12T02:10:41.189905+0000 mon.smithi153 (mon.0) 100 : audit 1 from='client.? 172.21.15.153:0/3940837048' entity='client.admin' 2024-03-12T02:10:48.751 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:10:50.484 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:50 smithi153 bash[14340]: cluster 2024-03-12T02:10:49.025295+0000 mgr.smithi153.asomxr (mgr.14164) 9 : cluster 0 pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:51.657 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:51 smithi153 bash[14340]: audit 2024-03-12T02:10:50.143446+0000 mon.smithi153 (mon.0) 101 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:51.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:51 smithi153 bash[14340]: audit 2024-03-12T02:10:50.149787+0000 mon.smithi153 (mon.0) 102 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:51.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:51 smithi153 bash[14340]: audit 2024-03-12T02:10:50.151397+0000 mon.smithi153 (mon.0) 103 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:10:51.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:51 smithi153 bash[14340]: audit 2024-03-12T02:10:50.156788+0000 mon.smithi153 (mon.0) 104 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:51.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:51 smithi153 bash[14340]: audit 2024-03-12T02:10:50.158691+0000 mon.smithi153 (mon.0) 105 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-12T02:10:51.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:51 smithi153 bash[14340]: audit 2024-03-12T02:10:50.162049+0000 mon.smithi153 (mon.0) 106 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-12T02:10:51.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:51 smithi153 bash[14340]: audit 2024-03-12T02:10:50.163244+0000 mon.smithi153 (mon.0) 107 : audit 0 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:10:51.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:51 smithi153 bash[14340]: cephadm 2024-03-12T02:10:50.164700+0000 mgr.smithi153.asomxr (mgr.14164) 10 : cephadm 1 Deploying daemon ceph-exporter.smithi153 on smithi153 2024-03-12T02:10:52.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:52 smithi153 bash[14340]: cluster 2024-03-12T02:10:51.025663+0000 mgr.smithi153.asomxr (mgr.14164) 11 : cluster 0 pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:53.636 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-12T02:10:53.637 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-12T02:10:53.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:53 smithi153 bash[14340]: audit 2024-03-12T02:10:52.772822+0000 mon.smithi153 (mon.0) 108 : audit 1 from='client.? 172.21.15.153:0/1203912064' entity='client.admin' 2024-03-12T02:10:53.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:53 smithi153 bash[14340]: cluster 2024-03-12T02:10:53.026073+0000 mgr.smithi153.asomxr (mgr.14164) 12 : cluster 0 pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:55.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:55 smithi153 bash[14340]: audit 2024-03-12T02:10:54.048557+0000 mon.smithi153 (mon.0) 109 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:55.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:55 smithi153 bash[14340]: audit 2024-03-12T02:10:54.053150+0000 mon.smithi153 (mon.0) 110 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:55.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:55 smithi153 bash[14340]: audit 2024-03-12T02:10:54.092858+0000 mon.smithi153 (mon.0) 111 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:55.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:55 smithi153 bash[14340]: audit 2024-03-12T02:10:54.097012+0000 mon.smithi153 (mon.0) 112 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:55.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:55 smithi153 bash[14340]: audit 2024-03-12T02:10:54.099167+0000 mon.smithi153 (mon.0) 113 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-12T02:10:55.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:55 smithi153 bash[14340]: audit 2024-03-12T02:10:54.101826+0000 mon.smithi153 (mon.0) 114 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-12T02:10:55.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:55 smithi153 bash[14340]: audit 2024-03-12T02:10:54.102744+0000 mon.smithi153 (mon.0) 115 : audit 0 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:10:55.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:55 smithi153 bash[14340]: cephadm 2024-03-12T02:10:54.103657+0000 mgr.smithi153.asomxr (mgr.14164) 13 : cephadm 1 Deploying daemon crash.smithi153 on smithi153 2024-03-12T02:10:55.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:55 smithi153 bash[14340]: audit 2024-03-12T02:10:54.112535+0000 mon.smithi153 (mon.0) 116 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:56.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:56 smithi153 bash[14340]: cluster 2024-03-12T02:10:55.026543+0000 mgr.smithi153.asomxr (mgr.14164) 14 : cluster 0 pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:58.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:57 smithi153 bash[14340]: audit 2024-03-12T02:10:56.945065+0000 mon.smithi153 (mon.0) 117 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:58.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:57 smithi153 bash[14340]: audit 2024-03-12T02:10:56.949701+0000 mon.smithi153 (mon.0) 118 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:58.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:57 smithi153 bash[14340]: audit 2024-03-12T02:10:56.977352+0000 mon.smithi153 (mon.0) 119 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:58.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:57 smithi153 bash[14340]: audit 2024-03-12T02:10:56.980991+0000 mon.smithi153 (mon.0) 120 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:58.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:57 smithi153 bash[14340]: cephadm 2024-03-12T02:10:56.982508+0000 mgr.smithi153.asomxr (mgr.14164) 15 : cephadm 1 Deploying daemon node-exporter.smithi153 on smithi153 2024-03-12T02:10:58.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:57 smithi153 bash[14340]: cluster 2024-03-12T02:10:57.026860+0000 mgr.smithi153.asomxr (mgr.14164) 16 : cluster 0 pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:10:59.494 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:59 smithi153 bash[14340]: audit 2024-03-12T02:10:58.142805+0000 mon.smithi153 (mon.0) 121 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:59.495 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:59 smithi153 bash[14340]: audit 2024-03-12T02:10:58.148985+0000 mon.smithi153 (mon.0) 122 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:59.495 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:59 smithi153 bash[14340]: audit 2024-03-12T02:10:58.206475+0000 mon.smithi153 (mon.0) 123 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:59.495 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:59 smithi153 bash[14340]: audit 2024-03-12T02:10:58.213731+0000 mon.smithi153 (mon.0) 124 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:10:59.495 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:59 smithi153 bash[14340]: cephadm 2024-03-12T02:10:58.232055+0000 mgr.smithi153.asomxr (mgr.14164) 17 : cephadm 1 Deploying daemon alertmanager.smithi153 on smithi153 2024-03-12T02:10:59.495 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:10:59 smithi153 bash[14340]: audit 2024-03-12T02:10:59.118379+0000 mon.smithi153 (mon.0) 125 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:00.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:00 smithi153 bash[14340]: cluster 2024-03-12T02:10:59.027174+0000 mgr.smithi153.asomxr (mgr.14164) 18 : cluster 0 pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:00.354 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:11:02.620 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:02 smithi153 bash[14340]: cluster 2024-03-12T02:11:01.027615+0000 mgr.smithi153.asomxr (mgr.14164) 19 : cluster 0 pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:03 smithi153 bash[14340]: cluster 2024-03-12T02:11:03.028033+0000 mgr.smithi153.asomxr (mgr.14164) 20 : cluster 0 pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:06.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:06 smithi153 bash[14340]: cluster 2024-03-12T02:11:05.028298+0000 mgr.smithi153.asomxr (mgr.14164) 21 : cluster 0 pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:07.654 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi163 2024-03-12T02:11:07.654 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T02:11:07.655 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.conf 2024-03-12T02:11:07.663 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T02:11:07.663 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T02:11:07.715 INFO:tasks.cephadm:Adding host smithi163 to orchestrator... 2024-03-12T02:11:07.716 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch host add smithi163 2024-03-12T02:11:07.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:07 smithi153 bash[14340]: audit 2024-03-12T02:11:06.629086+0000 mgr.smithi153.asomxr (mgr.14164) 22 : audit 0 from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:11:07.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:07 smithi153 bash[14340]: audit 2024-03-12T02:11:06.633558+0000 mon.smithi153 (mon.0) 126 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:07.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:07 smithi153 bash[14340]: cluster 2024-03-12T02:11:07.028656+0000 mgr.smithi153.asomxr (mgr.14164) 23 : cluster 0 pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:10.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:10 smithi153 bash[14340]: cluster 2024-03-12T02:11:09.029049+0000 mgr.smithi153.asomxr (mgr.14164) 24 : cluster 0 pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:10.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:10 smithi153 bash[14340]: audit 2024-03-12T02:11:10.050405+0000 mon.smithi153 (mon.0) 127 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:10.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:10 smithi153 bash[14340]: audit 2024-03-12T02:11:10.057488+0000 mon.smithi153 (mon.0) 128 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:10.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:10 smithi153 bash[14340]: audit 2024-03-12T02:11:10.079633+0000 mon.smithi153 (mon.0) 129 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:11.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:11 smithi153 bash[14340]: audit 2024-03-12T02:11:10.085371+0000 mon.smithi153 (mon.0) 130 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:11.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:11 smithi153 bash[14340]: audit 2024-03-12T02:11:10.096204+0000 mon.smithi153 (mon.0) 131 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:11.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:11 smithi153 bash[14340]: audit 2024-03-12T02:11:10.101349+0000 mon.smithi153 (mon.0) 132 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:11.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:11 smithi153 bash[14340]: cephadm 2024-03-12T02:11:10.114419+0000 mgr.smithi153.asomxr (mgr.14164) 25 : cephadm 1 Regenerating cephadm self-signed grafana TLS certificates 2024-03-12T02:11:11.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:11 smithi153 bash[14340]: audit 2024-03-12T02:11:10.160793+0000 mon.smithi153 (mon.0) 133 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:11.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:11 smithi153 bash[14340]: audit 2024-03-12T02:11:10.166040+0000 mon.smithi153 (mon.0) 134 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:11.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:11 smithi153 bash[14340]: audit 2024-03-12T02:11:10.167761+0000 mon.smithi153 (mon.0) 135 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-12T02:11:11.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:11 smithi153 bash[14340]: audit 2024-03-12T02:11:10.168278+0000 mgr.smithi153.asomxr (mgr.14164) 26 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-12T02:11:11.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:11 smithi153 bash[14340]: audit 2024-03-12T02:11:10.173718+0000 mon.smithi153 (mon.0) 136 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:11.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:11 smithi153 bash[14340]: cephadm 2024-03-12T02:11:10.185217+0000 mgr.smithi153.asomxr (mgr.14164) 27 : cephadm 1 Deploying daemon grafana.smithi153 on smithi153 2024-03-12T02:11:12.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:12 smithi153 bash[14340]: cluster 2024-03-12T02:11:11.029413+0000 mgr.smithi153.asomxr (mgr.14164) 28 : cluster 0 pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:13.407 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:11:13.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:13 smithi153 bash[14340]: cluster 2024-03-12T02:11:13.029807+0000 mgr.smithi153.asomxr (mgr.14164) 29 : cluster 0 pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:15.454 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:15 smithi153 bash[14340]: audit 2024-03-12T02:11:14.124075+0000 mon.smithi153 (mon.0) 137 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:16.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:16 smithi153 bash[14340]: cluster 2024-03-12T02:11:15.030084+0000 mgr.smithi153.asomxr (mgr.14164) 30 : cluster 0 pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:18.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:18 smithi153 bash[14340]: cluster 2024-03-12T02:11:17.030430+0000 mgr.smithi153.asomxr (mgr.14164) 31 : cluster 0 pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:20.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:20 smithi153 bash[14340]: cluster 2024-03-12T02:11:19.030821+0000 mgr.smithi153.asomxr (mgr.14164) 32 : cluster 0 pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:22.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:22 smithi153 bash[14340]: cluster 2024-03-12T02:11:21.031173+0000 mgr.smithi153.asomxr (mgr.14164) 33 : cluster 0 pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:23.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:23 smithi153 bash[14340]: audit 2024-03-12T02:11:21.821804+0000 mgr.smithi153.asomxr (mgr.14164) 34 : audit 0 from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi163", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:11:24.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:24 smithi153 bash[14340]: cephadm 2024-03-12T02:11:22.611889+0000 mgr.smithi153.asomxr (mgr.14164) 35 : cephadm 1 Deploying cephadm binary to smithi163 2024-03-12T02:11:24.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:24 smithi153 bash[14340]: cluster 2024-03-12T02:11:23.031541+0000 mgr.smithi153.asomxr (mgr.14164) 36 : cluster 0 pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:25.428 INFO:teuthology.orchestra.run.smithi153.stdout:Added host 'smithi163' with addr '172.21.15.163' 2024-03-12T02:11:26.717 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:26 smithi153 bash[14340]: cluster 2024-03-12T02:11:25.031958+0000 mgr.smithi153.asomxr (mgr.14164) 37 : cluster 0 pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:26.717 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:26 smithi153 bash[14340]: audit 2024-03-12T02:11:25.402090+0000 mon.smithi153 (mon.0) 138 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:27.257 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch host ls --format=json 2024-03-12T02:11:27.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:27 smithi153 bash[14340]: cephadm 2024-03-12T02:11:25.426390+0000 mgr.smithi153.asomxr (mgr.14164) 38 : cephadm 1 Added host smithi163 2024-03-12T02:11:28.570 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:28 smithi153 bash[14340]: cluster 2024-03-12T02:11:27.032446+0000 mgr.smithi153.asomxr (mgr.14164) 39 : cluster 0 pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:30.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:30 smithi153 bash[14340]: cluster 2024-03-12T02:11:29.032916+0000 mgr.smithi153.asomxr (mgr.14164) 40 : cluster 0 pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:31.992 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:31 smithi153 bash[14340]: audit 2024-03-12T02:11:30.710946+0000 mon.smithi153 (mon.0) 139 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:31.992 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:31 smithi153 bash[14340]: audit 2024-03-12T02:11:30.714875+0000 mon.smithi153 (mon.0) 140 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:31.992 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:31 smithi153 bash[14340]: audit 2024-03-12T02:11:30.765371+0000 mon.smithi153 (mon.0) 141 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:31.992 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:31 smithi153 bash[14340]: audit 2024-03-12T02:11:30.770444+0000 mon.smithi153 (mon.0) 142 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:31.993 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:31 smithi153 bash[14340]: audit 2024-03-12T02:11:30.773343+0000 mon.smithi153 (mon.0) 143 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:31.993 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:31 smithi153 bash[14340]: audit 2024-03-12T02:11:30.778593+0000 mon.smithi153 (mon.0) 144 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:31.993 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:31 smithi153 bash[14340]: audit 2024-03-12T02:11:30.781679+0000 mon.smithi153 (mon.0) 145 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:31.993 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:31 smithi153 bash[14340]: audit 2024-03-12T02:11:30.783838+0000 mon.smithi153 (mon.0) 146 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:31.993 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:31 smithi153 bash[14340]: cephadm 2024-03-12T02:11:31.022274+0000 mgr.smithi153.asomxr (mgr.14164) 41 : cephadm 1 Deploying daemon prometheus.smithi153 on smithi153 2024-03-12T02:11:31.993 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:31 smithi153 bash[14340]: cluster 2024-03-12T02:11:31.033281+0000 mgr.smithi153.asomxr (mgr.14164) 42 : cluster 0 pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:33.705 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:33 smithi153 bash[14340]: cluster 2024-03-12T02:11:33.033731+0000 mgr.smithi153.asomxr (mgr.14164) 43 : cluster 0 pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:34.146 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:11:35.397 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:35 smithi153 bash[14340]: audit 2024-03-12T02:11:34.135024+0000 mon.smithi153 (mon.0) 147 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:36.526 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:36 smithi153 bash[14340]: cluster 2024-03-12T02:11:35.034152+0000 mgr.smithi153.asomxr (mgr.14164) 44 : cluster 0 pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:38.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:38 smithi153 bash[14340]: cluster 2024-03-12T02:11:37.034453+0000 mgr.smithi153.asomxr (mgr.14164) 45 : cluster 0 pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:40.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:40 smithi153 bash[14340]: cluster 2024-03-12T02:11:39.034836+0000 mgr.smithi153.asomxr (mgr.14164) 46 : cluster 0 pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:41.651 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:11:41.651 INFO:teuthology.orchestra.run.smithi153.stdout:[{"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}, {"addr": "172.21.15.163", "hostname": "smithi163", "labels": [], "status": ""}] 2024-03-12T02:11:42.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:42 smithi153 bash[14340]: cluster 2024-03-12T02:11:41.035279+0000 mgr.smithi153.asomxr (mgr.14164) 47 : cluster 0 pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:42.929 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-12T02:11:42.929 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd crush tunables default 2024-03-12T02:11:43.218 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:43 smithi153 bash[14340]: audit 2024-03-12T02:11:41.649105+0000 mgr.smithi153.asomxr (mgr.14164) 48 : audit 0 from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:11:44.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:44 smithi153 bash[14340]: cluster 2024-03-12T02:11:43.035574+0000 mgr.smithi153.asomxr (mgr.14164) 49 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:46.215 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:46 smithi153 bash[14340]: cluster 2024-03-12T02:11:45.035952+0000 mgr.smithi153.asomxr (mgr.14164) 50 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:47.627 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:11:48.442 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:48 smithi153 bash[14340]: cluster 2024-03-12T02:11:47.036382+0000 mgr.smithi153.asomxr (mgr.14164) 51 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:50.469 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:50 smithi153 bash[14340]: cluster 2024-03-12T02:11:49.036768+0000 mgr.smithi153.asomxr (mgr.14164) 52 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:51.903 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:51 smithi153 bash[14340]: audit 2024-03-12T02:11:50.761434+0000 mon.smithi153 (mon.0) 148 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:51.904 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:51 smithi153 bash[14340]: audit 2024-03-12T02:11:50.765293+0000 mon.smithi153 (mon.0) 149 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:51.904 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:51 smithi153 bash[14340]: audit 2024-03-12T02:11:50.792094+0000 mon.smithi153 (mon.0) 150 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' 2024-03-12T02:11:51.904 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:51 smithi153 bash[14340]: audit 2024-03-12T02:11:50.794818+0000 mon.smithi153 (mon.0) 151 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-12T02:11:51.904 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:51 smithi153 bash[14340]: cluster 2024-03-12T02:11:51.037200+0000 mgr.smithi153.asomxr (mgr.14164) 53 : cluster 0 pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:11:52.937 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:52 smithi153 bash[14340]: audit 2024-03-12T02:11:51.796576+0000 mon.smithi153 (mon.0) 152 : audit 1 from='mgr.14164 172.21.15.153:0/3584414881' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-12T02:11:52.937 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:52 smithi153 bash[14340]: cluster 2024-03-12T02:11:51.796670+0000 mon.smithi153 (mon.0) 153 : cluster 0 mgrmap e14: smithi153.asomxr(active, since 82s) 2024-03-12T02:11:54.274 INFO:teuthology.orchestra.run.smithi153.stderr:adjusted tunables profile to default 2024-03-12T02:11:54.316 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:54 smithi153 bash[14340]: audit 2024-03-12T02:11:54.212772+0000 mon.smithi153 (mon.0) 154 : audit 1 from='client.? 172.21.15.153:0/2005011360' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-12T02:11:55.476 INFO:tasks.cephadm:Adding mon.smithi153 on smithi153 2024-03-12T02:11:55.476 INFO:tasks.cephadm:Adding mon.smithi163 on smithi163 2024-03-12T02:11:55.477 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch apply mon '2;smithi153:172.21.15.153=smithi153;smithi163:172.21.15.163=smithi163' 2024-03-12T02:11:55.522 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:55 smithi153 bash[14340]: audit 2024-03-12T02:11:54.267858+0000 mon.smithi153 (mon.0) 155 : audit 1 from='client.? 172.21.15.153:0/2005011360' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-12T02:11:55.522 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:11:55 smithi153 bash[14340]: cluster 2024-03-12T02:11:54.267972+0000 mon.smithi153 (mon.0) 156 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-03-12T02:11:56.664 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-12T02:11:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-12T02:12:00.524 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:00 smithi153 bash[14340]: cluster 2024-03-12T02:12:00.299098+0000 mon.smithi153 (mon.0) 157 : cluster 1 Active manager daemon smithi153.asomxr restarted 2024-03-12T02:12:00.524 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:00 smithi153 bash[14340]: cluster 2024-03-12T02:12:00.299572+0000 mon.smithi153 (mon.0) 158 : cluster 1 Activating manager daemon smithi153.asomxr 2024-03-12T02:12:00.524 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:00 smithi153 bash[14340]: cluster 2024-03-12T02:12:00.303498+0000 mon.smithi153 (mon.0) 159 : cluster 0 osdmap e5: 0 total, 0 up, 0 in 2024-03-12T02:12:00.524 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:00 smithi153 bash[14340]: cluster 2024-03-12T02:12:00.303641+0000 mon.smithi153 (mon.0) 160 : cluster 0 mgrmap e15: smithi153.asomxr(active, starting, since 0.00428034s) 2024-03-12T02:12:00.524 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:00 smithi153 bash[14340]: audit 2024-03-12T02:12:00.306279+0000 mon.smithi153 (mon.0) 161 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-12T02:12:00.524 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:00 smithi153 bash[14340]: audit 2024-03-12T02:12:00.307824+0000 mon.smithi153 (mon.0) 162 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr metadata", "who": "smithi153.asomxr", "id": "smithi153.asomxr"}]: dispatch 2024-03-12T02:12:00.524 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:00 smithi153 bash[14340]: audit 2024-03-12T02:12:00.308684+0000 mon.smithi153 (mon.0) 163 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-12T02:12:00.524 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:00 smithi153 bash[14340]: audit 2024-03-12T02:12:00.308859+0000 mon.smithi153 (mon.0) 164 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-12T02:12:00.525 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:00 smithi153 bash[14340]: audit 2024-03-12T02:12:00.309016+0000 mon.smithi153 (mon.0) 165 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-12T02:12:00.525 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:00 smithi153 bash[14340]: cluster 2024-03-12T02:12:00.316622+0000 mon.smithi153 (mon.0) 166 : cluster 1 Manager daemon smithi153.asomxr is now available 2024-03-12T02:12:01.316 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled mon update... 2024-03-12T02:12:01.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:01 smithi153 bash[14340]: audit 2024-03-12T02:12:00.351127+0000 mon.smithi153 (mon.0) 167 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:12:01.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:01 smithi153 bash[14340]: audit 2024-03-12T02:12:00.358368+0000 mon.smithi153 (mon.0) 168 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:12:01.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:01 smithi153 bash[14340]: audit 2024-03-12T02:12:00.362353+0000 mon.smithi153 (mon.0) 169 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:01.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:01 smithi153 bash[14340]: audit 2024-03-12T02:12:00.366900+0000 mon.smithi153 (mon.0) 170 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:12:01.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:01 smithi153 bash[14340]: audit 2024-03-12T02:12:00.376990+0000 mon.smithi153 (mon.0) 171 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.asomxr/mirror_snapshot_schedule"}]: dispatch 2024-03-12T02:12:01.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:01 smithi153 bash[14340]: audit 2024-03-12T02:12:00.386919+0000 mon.smithi153 (mon.0) 172 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.asomxr/trash_purge_schedule"}]: dispatch 2024-03-12T02:12:01.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:01 smithi153 bash[14340]: audit 2024-03-12T02:12:01.167468+0000 mon.smithi153 (mon.0) 173 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:01.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:01 smithi153 bash[14340]: cluster 2024-03-12T02:12:01.306773+0000 mon.smithi153 (mon.0) 174 : cluster 0 mgrmap e16: smithi153.asomxr(active, since 1.00741s) 2024-03-12T02:12:01.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:01 smithi153 bash[14340]: audit 2024-03-12T02:12:01.313944+0000 mon.smithi153 (mon.0) 175 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:01.926 DEBUG:teuthology.orchestra.run.smithi163:mon.smithi163> sudo journalctl -f -n 0 -u ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mon.smithi163.service 2024-03-12T02:12:01.929 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-12T02:12:01.929 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-12T02:12:02.129 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-03-12T02:12:02.612 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:02 smithi153 bash[14340]: cephadm 2024-03-12T02:12:01.311537+0000 mgr.smithi153.asomxr (mgr.14195) 2 : cephadm 1 Saving service mon spec with placement smithi153:172.21.15.153=smithi153;smithi163:172.21.15.163=smithi163;count:2 2024-03-12T02:12:02.612 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:02 smithi153 bash[14340]: cephadm 2024-03-12T02:12:01.315798+0000 mgr.smithi153.asomxr (mgr.14195) 3 : cephadm 1 [12/Mar/2024:02:12:01] ENGINE Bus STARTING 2024-03-12T02:12:02.612 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:02 smithi153 bash[14340]: cephadm 2024-03-12T02:12:01.438206+0000 mgr.smithi153.asomxr (mgr.14195) 4 : cephadm 1 [12/Mar/2024:02:12:01] ENGINE Serving on https://172.21.15.153:7150 2024-03-12T02:12:02.612 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:02 smithi153 bash[14340]: cephadm 2024-03-12T02:12:01.540153+0000 mgr.smithi153.asomxr (mgr.14195) 5 : cephadm 1 [12/Mar/2024:02:12:01] ENGINE Serving on http://172.21.15.153:8765 2024-03-12T02:12:02.612 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:02 smithi153 bash[14340]: cephadm 2024-03-12T02:12:01.540317+0000 mgr.smithi153.asomxr (mgr.14195) 6 : cephadm 1 [12/Mar/2024:02:12:01] ENGINE Bus STARTED 2024-03-12T02:12:03.161 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-12T02:12:03.387 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:03 smithi153 bash[14340]: cluster 2024-03-12T02:12:02.362226+0000 mon.smithi153 (mon.0) 176 : cluster 0 mgrmap e17: smithi153.asomxr(active, since 2s) 2024-03-12T02:12:03.387 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:03 smithi153 bash[14340]: audit 2024-03-12T02:12:02.990374+0000 mon.smithi153 (mon.0) 177 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:04.219 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-12T02:12:05.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:04 smithi153 bash[14340]: audit 2024-03-12T02:12:03.780284+0000 mon.smithi153 (mon.0) 178 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:07.726 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-12T02:12:07.727 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"5cbf0f4e-e015-11ee-95c8-87774f69a715","modified":"2024-03-12T02:09:01.494884Z","created":"2024-03-12T02:09:01.494884Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-12T02:12:07.730 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-03-12T02:12:08.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:07 smithi153 bash[14340]: audit 2024-03-12T02:12:06.786147+0000 mon.smithi153 (mon.0) 179 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:08.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:07 smithi153 bash[14340]: audit 2024-03-12T02:12:06.792550+0000 mon.smithi153 (mon.0) 180 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:08.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:07 smithi153 bash[14340]: audit 2024-03-12T02:12:07.582221+0000 mon.smithi153 (mon.0) 181 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:08.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:07 smithi153 bash[14340]: audit 2024-03-12T02:12:07.586924+0000 mon.smithi153 (mon.0) 182 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:08.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:07 smithi153 bash[14340]: audit 2024-03-12T02:12:07.588469+0000 mon.smithi153 (mon.0) 183 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:12:08.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:07 smithi153 bash[14340]: audit 2024-03-12T02:12:07.725018+0000 mon.smithi153 (mon.0) 184 : audit 0 from='client.? 172.21.15.163:0/2569240848' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T02:12:09.481 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-12T02:12:09.481 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-12T02:12:12.161 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T02:12:12.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.277168+0000 mon.smithi153 (mon.0) 185 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:12.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.281161+0000 mon.smithi153 (mon.0) 186 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:12.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.286236+0000 mon.smithi153 (mon.0) 187 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:12.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.290140+0000 mon.smithi153 (mon.0) 188 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:12.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.291273+0000 mon.smithi153 (mon.0) 189 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:12:12.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.292744+0000 mon.smithi153 (mon.0) 190 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:12.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.294078+0000 mon.smithi153 (mon.0) 191 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:12:12.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: cephadm 2024-03-12T02:12:11.295952+0000 mgr.smithi153.asomxr (mgr.14195) 7 : cephadm 1 Updating smithi153:/etc/ceph/ceph.conf 2024-03-12T02:12:12.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: cephadm 2024-03-12T02:12:11.296251+0000 mgr.smithi153.asomxr (mgr.14195) 8 : cephadm 1 Updating smithi163:/etc/ceph/ceph.conf 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: cephadm 2024-03-12T02:12:11.461669+0000 mgr.smithi153.asomxr (mgr.14195) 9 : cephadm 1 Updating smithi153:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: cephadm 2024-03-12T02:12:11.462015+0000 mgr.smithi153.asomxr (mgr.14195) 10 : cephadm 1 Updating smithi163:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: cephadm 2024-03-12T02:12:11.626869+0000 mgr.smithi153.asomxr (mgr.14195) 11 : cephadm 1 Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: cephadm 2024-03-12T02:12:11.627241+0000 mgr.smithi153.asomxr (mgr.14195) 12 : cephadm 1 Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: cephadm 2024-03-12T02:12:11.792170+0000 mgr.smithi153.asomxr (mgr.14195) 13 : cephadm 1 Updating smithi163:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: cephadm 2024-03-12T02:12:11.793266+0000 mgr.smithi153.asomxr (mgr.14195) 14 : cephadm 1 Updating smithi153:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.962662+0000 mon.smithi153 (mon.0) 192 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.965293+0000 mon.smithi153 (mon.0) 193 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.968946+0000 mon.smithi153 (mon.0) 194 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.973380+0000 mon.smithi153 (mon.0) 195 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.977702+0000 mon.smithi153 (mon.0) 196 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.979406+0000 mon.smithi153 (mon.0) 197 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi163", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.982164+0000 mon.smithi153 (mon.0) 198 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi163", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-12T02:12:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:12 smithi153 bash[14340]: audit 2024-03-12T02:12:11.983162+0000 mon.smithi153 (mon.0) 199 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:13.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:13 smithi153 bash[14340]: cephadm 2024-03-12T02:12:11.984434+0000 mgr.smithi153.asomxr (mgr.14195) 15 : cephadm 1 Deploying daemon ceph-exporter.smithi163 on smithi163 2024-03-12T02:12:15.577 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-12T02:12:15.578 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"5cbf0f4e-e015-11ee-95c8-87774f69a715","modified":"2024-03-12T02:09:01.494884Z","created":"2024-03-12T02:09:01.494884Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-12T02:12:15.580 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-03-12T02:12:15.776 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:15 smithi153 bash[14340]: audit 2024-03-12T02:12:15.367246+0000 mon.smithi153 (mon.0) 200 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:12:16.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:16 smithi153 bash[14340]: audit 2024-03-12T02:12:15.497407+0000 mon.smithi153 (mon.0) 201 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:16.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:16 smithi153 bash[14340]: audit 2024-03-12T02:12:15.503474+0000 mon.smithi153 (mon.0) 202 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:16.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:16 smithi153 bash[14340]: audit 2024-03-12T02:12:15.513676+0000 mon.smithi153 (mon.0) 203 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:16.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:16 smithi153 bash[14340]: audit 2024-03-12T02:12:15.520264+0000 mon.smithi153 (mon.0) 204 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:16.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:16 smithi153 bash[14340]: audit 2024-03-12T02:12:15.522296+0000 mon.smithi153 (mon.0) 205 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-12T02:12:16.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:16 smithi153 bash[14340]: audit 2024-03-12T02:12:15.527577+0000 mon.smithi153 (mon.0) 206 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-12T02:12:16.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:16 smithi153 bash[14340]: audit 2024-03-12T02:12:15.528975+0000 mon.smithi153 (mon.0) 207 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:16.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:16 smithi153 bash[14340]: cephadm 2024-03-12T02:12:15.530734+0000 mgr.smithi153.asomxr (mgr.14195) 16 : cephadm 1 Deploying daemon crash.smithi163 on smithi163 2024-03-12T02:12:16.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:16 smithi153 bash[14340]: audit 2024-03-12T02:12:15.576005+0000 mon.smithi153 (mon.0) 208 : audit 0 from='client.? 172.21.15.163:0/408540913' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T02:12:17.710 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-12T02:12:17.710 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-12T02:12:19.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:19 smithi153 bash[14340]: audit 2024-03-12T02:12:18.341124+0000 mon.smithi153 (mon.0) 209 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:19.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:19 smithi153 bash[14340]: audit 2024-03-12T02:12:18.347576+0000 mon.smithi153 (mon.0) 210 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:19.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:19 smithi153 bash[14340]: audit 2024-03-12T02:12:18.359271+0000 mon.smithi153 (mon.0) 211 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:19.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:19 smithi153 bash[14340]: audit 2024-03-12T02:12:18.365845+0000 mon.smithi153 (mon.0) 212 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:19.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:19 smithi153 bash[14340]: cephadm 2024-03-12T02:12:18.368461+0000 mgr.smithi153.asomxr (mgr.14195) 17 : cephadm 1 Deploying daemon node-exporter.smithi163 on smithi163 2024-03-12T02:12:20.791 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:20 smithi153 bash[14340]: audit 2024-03-12T02:12:19.386433+0000 mon.smithi153 (mon.0) 213 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:20.791 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:20 smithi153 bash[14340]: audit 2024-03-12T02:12:19.392045+0000 mon.smithi153 (mon.0) 214 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:20.791 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:20 smithi153 bash[14340]: audit 2024-03-12T02:12:19.440482+0000 mon.smithi153 (mon.0) 215 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:20.791 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:20 smithi153 bash[14340]: audit 2024-03-12T02:12:19.446160+0000 mon.smithi153 (mon.0) 216 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:20.791 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:20 smithi153 bash[14340]: audit 2024-03-12T02:12:19.450378+0000 mon.smithi153 (mon.0) 217 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.fvmiae", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T02:12:20.791 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:20 smithi153 bash[14340]: audit 2024-03-12T02:12:19.456202+0000 mon.smithi153 (mon.0) 218 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.fvmiae", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T02:12:20.792 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:20 smithi153 bash[14340]: audit 2024-03-12T02:12:19.457655+0000 mon.smithi153 (mon.0) 219 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T02:12:20.792 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:20 smithi153 bash[14340]: audit 2024-03-12T02:12:19.459259+0000 mon.smithi153 (mon.0) 220 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:20.792 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:20 smithi153 bash[14340]: cephadm 2024-03-12T02:12:19.461022+0000 mgr.smithi153.asomxr (mgr.14195) 18 : cephadm 1 Deploying daemon mgr.smithi163.fvmiae on smithi163 2024-03-12T02:12:20.792 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:20 smithi153 bash[14340]: audit 2024-03-12T02:12:20.366174+0000 mon.smithi153 (mon.0) 221 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:21.365 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T02:12:21.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:21 smithi153 bash[14340]: cluster 2024-03-12T02:12:20.310407+0000 mgr.smithi153.asomxr (mgr.14195) 19 : cluster 0 pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:23.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:23 smithi153 bash[14340]: cluster 2024-03-12T02:12:22.310928+0000 mgr.smithi153.asomxr (mgr.14195) 20 : cluster 0 pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:23.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:23 smithi153 bash[14340]: audit 2024-03-12T02:12:22.951898+0000 mon.smithi153 (mon.0) 222 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:23.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:23 smithi153 bash[14340]: audit 2024-03-12T02:12:22.958858+0000 mon.smithi153 (mon.0) 223 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:23.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:23 smithi153 bash[14340]: audit 2024-03-12T02:12:22.990790+0000 mon.smithi153 (mon.0) 224 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:23.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:23 smithi153 bash[14340]: audit 2024-03-12T02:12:22.996624+0000 mon.smithi153 (mon.0) 225 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:23.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:23 smithi153 bash[14340]: audit 2024-03-12T02:12:22.998627+0000 mon.smithi153 (mon.0) 226 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T02:12:23.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:23 smithi153 bash[14340]: audit 2024-03-12T02:12:23.000017+0000 mon.smithi153 (mon.0) 227 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:24.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:24 smithi153 bash[14340]: cephadm 2024-03-12T02:12:23.001548+0000 mgr.smithi153.asomxr (mgr.14195) 21 : cephadm 1 Deploying daemon mon.smithi163 on smithi163 2024-03-12T02:12:25.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:25 smithi153 bash[14340]: cluster 2024-03-12T02:12:24.311358+0000 mgr.smithi153.asomxr (mgr.14195) 22 : cluster 0 pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:25.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:25 smithi153 bash[14340]: audit 2024-03-12T02:12:25.373238+0000 mon.smithi153 (mon.0) 228 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:26.559 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-12T02:12:26.559 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":1,"fsid":"5cbf0f4e-e015-11ee-95c8-87774f69a715","modified":"2024-03-12T02:09:01.494884Z","created":"2024-03-12T02:09:01.494884Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-12T02:12:26.561 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 1 2024-03-12T02:12:27.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:27 smithi153 bash[14340]: cluster 2024-03-12T02:12:26.311913+0000 mgr.smithi153.asomxr (mgr.14195) 23 : cluster 0 pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:27.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:27 smithi153 bash[14340]: audit 2024-03-12T02:12:26.556960+0000 mon.smithi153 (mon.0) 229 : audit 0 from='client.? 172.21.15.163:0/2453319013' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T02:12:28.533 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-12T02:12:28.533 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-12T02:12:29.281 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:29 smithi163 systemd[1]: Started Ceph mon.smithi163 for 5cbf0f4e-e015-11ee-95c8-87774f69a715. 2024-03-12T02:12:29.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:29 smithi153 bash[14340]: cluster 2024-03-12T02:12:28.312336+0000 mgr.smithi153.asomxr (mgr.14195) 24 : cluster 0 pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:29.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:29 smithi153 bash[14340]: audit 2024-03-12T02:12:29.278471+0000 mon.smithi153 (mon.0) 230 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:29.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:29 smithi153 bash[14340]: audit 2024-03-12T02:12:29.283523+0000 mon.smithi153 (mon.0) 231 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:29.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:29 smithi153 bash[14340]: audit 2024-03-12T02:12:29.338832+0000 mon.smithi153 (mon.0) 232 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:29.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:29 smithi153 bash[14340]: audit 2024-03-12T02:12:29.344705+0000 mon.smithi153 (mon.0) 233 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:29.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:29 smithi153 bash[14340]: audit 2024-03-12T02:12:29.350172+0000 mon.smithi153 (mon.0) 234 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:29.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:29 smithi153 bash[14340]: audit 2024-03-12T02:12:29.415444+0000 mon.smithi153 (mon.0) 235 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:12:30.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.206+0000 7f42ca3d0c80 0 set uid:gid to 167:167 (ceph:ceph) 2024-03-12T02:12:30.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.206+0000 7f42ca3d0c80 0 ceph version 18.2.1-746-gc2ea977b (c2ea977b6956d1f694ccf66acc692fdd979e3baa) reef (stable), process ceph-mon, pid 7 2024-03-12T02:12:30.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.206+0000 7f42ca3d0c80 0 pidfile_write: ignore empty --pid-file 2024-03-12T02:12:30.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 0 load: jerasure load: lrc 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: RocksDB version: 7.9.2 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Git sha 0 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Compile date 2024-03-09 15:03:01 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: DB SUMMARY 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: DB Session ID: QTVW5UPSQQDBPCOY4KZ2 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: CURRENT file: CURRENT 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: IDENTITY file: IDENTITY 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi163/store.db dir, Total Num: 0, files: 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi163/store.db: 000004.log size: 511 ; 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.error_if_exists: 0 2024-03-12T02:12:30.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.create_if_missing: 0 2024-03-12T02:12:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.paranoid_checks: 1 2024-03-12T02:12:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-03-12T02:12:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-12T02:12:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-12T02:12:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.env: 0x55cc9c0b4720 2024-03-12T02:12:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.fs: PosixFileSystem 2024-03-12T02:12:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.info_log: 0x55cc9dadf3c0 2024-03-12T02:12:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_file_opening_threads: 16 2024-03-12T02:12:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.statistics: (nil) 2024-03-12T02:12:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.use_fsync: 0 2024-03-12T02:12:30.488 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_log_file_size: 0 2024-03-12T02:12:30.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-12T02:12:30.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-03-12T02:12:30.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.keep_log_file_num: 1000 2024-03-12T02:12:30.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.recycle_log_file_num: 0 2024-03-12T02:12:30.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.allow_fallocate: 1 2024-03-12T02:12:30.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.allow_mmap_reads: 0 2024-03-12T02:12:30.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.allow_mmap_writes: 0 2024-03-12T02:12:30.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.use_direct_reads: 0 2024-03-12T02:12:30.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-12T02:12:30.490 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.create_missing_column_families: 0 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.db_log_dir: 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.wal_dir: 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.advise_random_on_open: 1 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.db_write_buffer_size: 0 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.write_buffer_manager: 0x55cc9cd74320 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-12T02:12:30.491 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.rate_limiter: (nil) 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.wal_recovery_mode: 2 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.enable_thread_tracking: 0 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.enable_pipelined_write: 0 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.unordered_write: 0 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.row_cache: None 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.wal_filter: None 2024-03-12T02:12:30.492 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.allow_ingest_behind: 0 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.two_write_queues: 0 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.manual_wal_flush: 0 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.wal_compression: 0 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.atomic_flush: 0 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.log_readahead_size: 0 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.best_efforts_recovery: 0 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-12T02:12:30.493 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.allow_data_in_errors: 0 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.db_host_id: __hostname__ 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_background_jobs: 2 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_background_compactions: -1 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_subcompactions: 1 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_total_wal_size: 0 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-03-12T02:12:30.494 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_open_files: -1 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bytes_per_sync: 0 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_readahead_size: 0 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_background_flushes: -1 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Compression algorithms supported: 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: kZSTD supported: 0 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: kXpressCompression supported: 0 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: kLZ4HCCompression supported: 1 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: kZlibCompression supported: 1 2024-03-12T02:12:30.495 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: kSnappyCompression supported: 1 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: kLZ4Compression supported: 1 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: kBZip2Compression supported: 0 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi163/store.db/MANIFEST-000005 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.merge_operator: 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_filter: None 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_filter_factory: None 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.sst_partitioner_factory: None 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cc9dadf4e0) 2024-03-12T02:12:30.496 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cache_index_and_filter_blocks: 1 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: pin_top_level_index_and_filter: 1 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: index_type: 0 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: data_block_index_type: 0 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: index_shortening: 1 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: data_block_hash_table_util_ratio: 0.750000 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: checksum: 4 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: no_block_cache: 0 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: block_cache: 0x55cc9cdf71f0 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: block_cache_name: BinnedLRUCache 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: block_cache_options: 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: capacity : 536870912 2024-03-12T02:12:30.497 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: num_shard_bits : 4 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: strict_capacity_limit : 0 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: high_pri_pool_ratio: 0.000 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: block_cache_compressed: (nil) 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: persistent_cache: (nil) 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: block_size: 4096 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: block_size_deviation: 10 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: block_restart_interval: 16 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: index_block_restart_interval: 1 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: metadata_block_size: 4096 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: partition_filters: 0 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: use_delta_encoding: 1 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: filter_policy: bloomfilter 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: whole_key_filtering: 1 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: verify_compression: 0 2024-03-12T02:12:30.498 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: read_amp_bytes_per_bit: 0 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: format_version: 5 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: enable_index_compression: 1 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: block_align: 0 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: max_auto_readahead_size: 262144 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: prepopulate_block_cache: 0 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: initial_auto_readahead_size: 8192 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: num_file_reads_for_auto_readahead: 2 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.write_buffer_size: 33554432 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_write_buffer_number: 2 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compression: NoCompression 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bottommost_compression: Disabled 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.prefix_extractor: nullptr 2024-03-12T02:12:30.499 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.num_levels: 7 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-12T02:12:30.500 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compression_opts.level: 32767 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compression_opts.strategy: 0 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compression_opts.enabled: false 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.target_file_size_base: 67108864 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-03-12T02:12:30.501 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.arena_block_size: 1048576 2024-03-12T02:12:30.502 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.disable_auto_compactions: 0 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.table_properties_collectors: 2024-03-12T02:12:30.503 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.inplace_update_support: 0 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.bloom_locality: 0 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.max_successive_merges: 0 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.paranoid_file_checks: 0 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.force_consistency_checks: 1 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.report_bg_io_stats: 0 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.ttl: 2592000 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-12T02:12:30.504 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.enable_blob_files: false 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.min_blob_size: 0 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.blob_file_size: 268435456 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.blob_file_starting_level: 0 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.210+0000 7f42ca3d0c80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi163/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 19feb75a-83d4-406b-be16-cd057cdab0ef 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1710209550218554, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-12T02:12:30.505 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-12T02:12:30.506 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1710209550219093, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710209550, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "19feb75a-83d4-406b-be16-cd057cdab0ef", "db_session_id": "QTVW5UPSQQDBPCOY4KZ2", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-12T02:12:30.506 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1710209550219201, "job": 1, "event": "recovery_finished"} 2024-03-12T02:12:30.506 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-12T02:12:30.506 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 4 rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi163/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-12T02:12:30.506 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cc9ce98000 2024-03-12T02:12:30.507 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 4 rocksdb: DB pointer 0x55cc9ce86000 2024-03-12T02:12:30.507 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42b8bca700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-12T02:12:30.507 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42b8bca700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-12T02:12:30.507 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: ** DB Stats ** 2024-03-12T02:12:30.507 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T02:12:30.507 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-12T02:12:30.507 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T02:12:30.507 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T02:12:30.507 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-12T02:12:30.507 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: ** Compaction Stats [default] ** 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: ** Compaction Stats [default] ** 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-12T02:12:30.508 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Uptime(secs): 0.0 total, 0.0 interval 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Flush(GB): cumulative 0.000, interval 0.000 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: AddFile(Total Files): cumulative 0, interval 0 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: AddFile(L0 Files): cumulative 0, interval 0 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: AddFile(Keys): cumulative 0, interval 0 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Cumulative compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Interval compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Block cache BinnedLRUCache@0x55cc9cdf71f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: ** File Read Latency Histogram By Level [default] ** 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 0 mon.smithi163 does not exist in monmap, will attempt to join an existing cluster 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.214+0000 7f42ca3d0c80 0 using public_addr v2:172.21.15.163:0/0 -> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.218+0000 7f42ca3d0c80 0 starting mon.smithi163 rank -1 at public addrs [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] at bind addrs [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi163 fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:12:30.509 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.218+0000 7f42ca3d0c80 1 mon.smithi163@-1(???) e0 preinit fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.250+0000 7f42bbbd0700 0 mon.smithi163@-1(synchronizing).mds e1 new map 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.250+0000 7f42bbbd0700 0 mon.smithi163@-1(synchronizing).mds e1 print_map 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: e1 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: enable_multiple, ever_enabled_multiple: 1,1 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: legacy client fscid: -1 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: No filesystems configured 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.250+0000 7f42bbbd0700 1 mon.smithi163@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.250+0000 7f42bbbd0700 1 mon.smithi163@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.250+0000 7f42bbbd0700 1 mon.smithi163@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.250+0000 7f42bbbd0700 1 mon.smithi163@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.250+0000 7f42bbbd0700 1 mon.smithi163@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-12T02:12:30.510 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.250+0000 7f42bbbd0700 1 mon.smithi163@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.250+0000 7f42bbbd0700 1 mon.smithi163@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.254+0000 7f42bbbd0700 0 mon.smithi163@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.254+0000 7f42bbbd0700 0 mon.smithi163@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.254+0000 7f42bbbd0700 0 mon.smithi163@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.254+0000 7f42bbbd0700 0 mon.smithi163@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:06.786147+0000 mon.smithi153 (mon.0) 179 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:06.792550+0000 mon.smithi153 (mon.0) 180 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:07.582221+0000 mon.smithi153 (mon.0) 181 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:07.586924+0000 mon.smithi153 (mon.0) 182 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:07.588469+0000 mon.smithi153 (mon.0) 183 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:07.725018+0000 mon.smithi153 (mon.0) 184 : audit 0 from='client.? 172.21.15.163:0/2569240848' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.277168+0000 mon.smithi153 (mon.0) 185 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.281161+0000 mon.smithi153 (mon.0) 186 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.511 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.286236+0000 mon.smithi153 (mon.0) 187 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.290140+0000 mon.smithi153 (mon.0) 188 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.291273+0000 mon.smithi153 (mon.0) 189 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi163", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.292744+0000 mon.smithi153 (mon.0) 190 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.294078+0000 mon.smithi153 (mon.0) 191 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:11.295952+0000 mgr.smithi153.asomxr (mgr.14195) 7 : cephadm 1 Updating smithi153:/etc/ceph/ceph.conf 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:11.296251+0000 mgr.smithi153.asomxr (mgr.14195) 8 : cephadm 1 Updating smithi163:/etc/ceph/ceph.conf 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:11.461669+0000 mgr.smithi153.asomxr (mgr.14195) 9 : cephadm 1 Updating smithi153:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:11.462015+0000 mgr.smithi153.asomxr (mgr.14195) 10 : cephadm 1 Updating smithi163:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:11.626869+0000 mgr.smithi153.asomxr (mgr.14195) 11 : cephadm 1 Updating smithi163:/etc/ceph/ceph.client.admin.keyring 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:11.627241+0000 mgr.smithi153.asomxr (mgr.14195) 12 : cephadm 1 Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:11.792170+0000 mgr.smithi153.asomxr (mgr.14195) 13 : cephadm 1 Updating smithi163:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:11.793266+0000 mgr.smithi153.asomxr (mgr.14195) 14 : cephadm 1 Updating smithi153:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.962662+0000 mon.smithi153 (mon.0) 192 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.512 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.965293+0000 mon.smithi153 (mon.0) 193 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.968946+0000 mon.smithi153 (mon.0) 194 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.973380+0000 mon.smithi153 (mon.0) 195 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.977702+0000 mon.smithi153 (mon.0) 196 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.979406+0000 mon.smithi153 (mon.0) 197 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi163", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.982164+0000 mon.smithi153 (mon.0) 198 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi163", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:11.983162+0000 mon.smithi153 (mon.0) 199 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:11.984434+0000 mgr.smithi153.asomxr (mgr.14195) 15 : cephadm 1 Deploying daemon ceph-exporter.smithi163 on smithi163 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:15.367246+0000 mon.smithi153 (mon.0) 200 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:15.497407+0000 mon.smithi153 (mon.0) 201 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:15.503474+0000 mon.smithi153 (mon.0) 202 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:15.513676+0000 mon.smithi153 (mon.0) 203 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:15.520264+0000 mon.smithi153 (mon.0) 204 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:15.522296+0000 mon.smithi153 (mon.0) 205 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-12T02:12:30.513 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:15.527577+0000 mon.smithi153 (mon.0) 206 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:15.528975+0000 mon.smithi153 (mon.0) 207 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:15.530734+0000 mgr.smithi153.asomxr (mgr.14195) 16 : cephadm 1 Deploying daemon crash.smithi163 on smithi163 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:15.576005+0000 mon.smithi153 (mon.0) 208 : audit 0 from='client.? 172.21.15.163:0/408540913' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:18.341124+0000 mon.smithi153 (mon.0) 209 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:18.347576+0000 mon.smithi153 (mon.0) 210 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:18.359271+0000 mon.smithi153 (mon.0) 211 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:18.365845+0000 mon.smithi153 (mon.0) 212 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:18.368461+0000 mgr.smithi153.asomxr (mgr.14195) 17 : cephadm 1 Deploying daemon node-exporter.smithi163 on smithi163 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:19.386433+0000 mon.smithi153 (mon.0) 213 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:19.392045+0000 mon.smithi153 (mon.0) 214 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:19.440482+0000 mon.smithi153 (mon.0) 215 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:19.446160+0000 mon.smithi153 (mon.0) 216 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:19.450378+0000 mon.smithi153 (mon.0) 217 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.fvmiae", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T02:12:30.514 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:19.456202+0000 mon.smithi153 (mon.0) 218 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.fvmiae", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:19.457655+0000 mon.smithi153 (mon.0) 219 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:19.459259+0000 mon.smithi153 (mon.0) 220 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:19.461022+0000 mgr.smithi153.asomxr (mgr.14195) 18 : cephadm 1 Deploying daemon mgr.smithi163.fvmiae on smithi163 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:20.366174+0000 mon.smithi153 (mon.0) 221 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cluster 2024-03-12T02:12:20.310407+0000 mgr.smithi153.asomxr (mgr.14195) 19 : cluster 0 pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cluster 2024-03-12T02:12:22.310928+0000 mgr.smithi153.asomxr (mgr.14195) 20 : cluster 0 pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:22.951898+0000 mon.smithi153 (mon.0) 222 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:22.958858+0000 mon.smithi153 (mon.0) 223 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:22.990790+0000 mon.smithi153 (mon.0) 224 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:22.996624+0000 mon.smithi153 (mon.0) 225 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:22.998627+0000 mon.smithi153 (mon.0) 226 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:23.000017+0000 mon.smithi153 (mon.0) 227 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cephadm 2024-03-12T02:12:23.001548+0000 mgr.smithi153.asomxr (mgr.14195) 21 : cephadm 1 Deploying daemon mon.smithi163 on smithi163 2024-03-12T02:12:30.515 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cluster 2024-03-12T02:12:24.311358+0000 mgr.smithi153.asomxr (mgr.14195) 22 : cluster 0 pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:30.516 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:25.373238+0000 mon.smithi153 (mon.0) 228 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.516 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cluster 2024-03-12T02:12:26.311913+0000 mgr.smithi153.asomxr (mgr.14195) 23 : cluster 0 pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:30.516 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:26.556960+0000 mon.smithi153 (mon.0) 229 : audit 0 from='client.? 172.21.15.163:0/2453319013' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T02:12:30.516 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: cluster 2024-03-12T02:12:28.312336+0000 mgr.smithi153.asomxr (mgr.14195) 24 : cluster 0 pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:30.516 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:29.278471+0000 mon.smithi153 (mon.0) 230 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.516 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:29.283523+0000 mon.smithi153 (mon.0) 231 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.516 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:29.338832+0000 mon.smithi153 (mon.0) 232 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.516 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:29.344705+0000 mon.smithi153 (mon.0) 233 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.516 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:29.350172+0000 mon.smithi153 (mon.0) 234 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:30.516 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: audit 2024-03-12T02:12:29.415444+0000 mon.smithi153 (mon.0) 235 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:12:30.516 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:30 smithi163 bash[22347]: debug 2024-03-12T02:12:30.254+0000 7f42bbbd0700 1 mon.smithi163@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-03-12T02:12:34.235 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi163/config 2024-03-12T02:12:35.404 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:30.274964+0000 mon.smithi153 (mon.0) 237 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-12T02:12:35.404 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:30.275247+0000 mon.smithi153 (mon.0) 238 : cluster 1 mon.smithi153 calling monitor election 2024-03-12T02:12:35.404 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:30.298423+0000 mon.smithi153 (mon.0) 239 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:30.312777+0000 mgr.smithi153.asomxr (mgr.14195) 25 : cluster 0 pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:30.367647+0000 mon.smithi153 (mon.0) 240 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:31.269923+0000 mon.smithi153 (mon.0) 241 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:32.269866+0000 mon.smithi163 (mon.1) 1 : cluster 1 mon.smithi163 calling monitor election 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:32.270146+0000 mon.smithi153 (mon.0) 242 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:32.313170+0000 mgr.smithi153.asomxr (mgr.14195) 26 : cluster 0 pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:33.270280+0000 mon.smithi153 (mon.0) 243 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:34.218265+0000 mon.smithi153 (mon.0) 244 : audit 0 from='mgr.? 172.21.15.163:0/2594050986' entity='mgr.smithi163.fvmiae' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.fvmiae/crt"}]: dispatch 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:34.270154+0000 mon.smithi153 (mon.0) 245 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:34.313535+0000 mgr.smithi153.asomxr (mgr.14195) 27 : cluster 0 pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:35.270325+0000 mon.smithi153 (mon.0) 246 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:35.318686+0000 mon.smithi153 (mon.0) 247 : cluster 1 mon.smithi153 is new leader, mons smithi153,smithi163 in quorum (ranks 0,1) 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:35.327414+0000 mon.smithi153 (mon.0) 248 : cluster 0 monmap e2: 2 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],smithi163=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T02:12:35.405 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:35.346076+0000 mon.smithi153 (mon.0) 249 : cluster 0 fsmap 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:35.346169+0000 mon.smithi153 (mon.0) 250 : cluster 0 osdmap e5: 0 total, 0 up, 0 in 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:35.346466+0000 mon.smithi153 (mon.0) 251 : cluster 0 mgrmap e17: smithi153.asomxr(active, since 35s) 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:35.349437+0000 mon.smithi153 (mon.0) 252 : cluster 0 Standby manager daemon smithi163.fvmiae started 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: cluster 2024-03-12T02:12:35.355052+0000 mon.smithi153 (mon.0) 253 : cluster 1 overall HEALTH_OK 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:35.355566+0000 mon.smithi153 (mon.0) 254 : audit 0 from='mgr.? 172.21.15.163:0/2594050986' entity='mgr.smithi163.fvmiae' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:35.358050+0000 mon.smithi153 (mon.0) 255 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:35.358766+0000 mon.smithi153 (mon.0) 256 : audit 0 from='mgr.? 172.21.15.163:0/2594050986' entity='mgr.smithi163.fvmiae' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.fvmiae/key"}]: dispatch 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:35.359453+0000 mon.smithi153 (mon.0) 257 : audit 0 from='mgr.? 172.21.15.163:0/2594050986' entity='mgr.smithi163.fvmiae' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:35.361245+0000 mon.smithi153 (mon.0) 258 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:35.367779+0000 mon.smithi153 (mon.0) 259 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:35.369032+0000 mon.smithi153 (mon.0) 260 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:35.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:35 smithi163 bash[22347]: audit 2024-03-12T02:12:35.370297+0000 mon.smithi153 (mon.0) 261 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:12:35.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:30.274964+0000 mon.smithi153 (mon.0) 237 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:30.275247+0000 mon.smithi153 (mon.0) 238 : cluster 1 mon.smithi153 calling monitor election 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:30.298423+0000 mon.smithi153 (mon.0) 239 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:30.312777+0000 mgr.smithi153.asomxr (mgr.14195) 25 : cluster 0 pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:30.367647+0000 mon.smithi153 (mon.0) 240 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:31.269923+0000 mon.smithi153 (mon.0) 241 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:32.269866+0000 mon.smithi163 (mon.1) 1 : cluster 1 mon.smithi163 calling monitor election 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:32.270146+0000 mon.smithi153 (mon.0) 242 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:32.313170+0000 mgr.smithi153.asomxr (mgr.14195) 26 : cluster 0 pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:33.270280+0000 mon.smithi153 (mon.0) 243 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:34.218265+0000 mon.smithi153 (mon.0) 244 : audit 0 from='mgr.? 172.21.15.163:0/2594050986' entity='mgr.smithi163.fvmiae' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.fvmiae/crt"}]: dispatch 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:34.270154+0000 mon.smithi153 (mon.0) 245 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:34.313535+0000 mgr.smithi153.asomxr (mgr.14195) 27 : cluster 0 pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:35.270325+0000 mon.smithi153 (mon.0) 246 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:35.318686+0000 mon.smithi153 (mon.0) 247 : cluster 1 mon.smithi153 is new leader, mons smithi153,smithi163 in quorum (ranks 0,1) 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:35.327414+0000 mon.smithi153 (mon.0) 248 : cluster 0 monmap e2: 2 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],smithi163=[v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:35.346076+0000 mon.smithi153 (mon.0) 249 : cluster 0 fsmap 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:35.346169+0000 mon.smithi153 (mon.0) 250 : cluster 0 osdmap e5: 0 total, 0 up, 0 in 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:35.346466+0000 mon.smithi153 (mon.0) 251 : cluster 0 mgrmap e17: smithi153.asomxr(active, since 35s) 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:35.349437+0000 mon.smithi153 (mon.0) 252 : cluster 0 Standby manager daemon smithi163.fvmiae started 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: cluster 2024-03-12T02:12:35.355052+0000 mon.smithi153 (mon.0) 253 : cluster 1 overall HEALTH_OK 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:35.355566+0000 mon.smithi153 (mon.0) 254 : audit 0 from='mgr.? 172.21.15.163:0/2594050986' entity='mgr.smithi163.fvmiae' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:35.358050+0000 mon.smithi153 (mon.0) 255 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:35.358766+0000 mon.smithi153 (mon.0) 256 : audit 0 from='mgr.? 172.21.15.163:0/2594050986' entity='mgr.smithi163.fvmiae' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi163.fvmiae/key"}]: dispatch 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:35.359453+0000 mon.smithi153 (mon.0) 257 : audit 0 from='mgr.? 172.21.15.163:0/2594050986' entity='mgr.smithi163.fvmiae' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-12T02:12:35.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:35.361245+0000 mon.smithi153 (mon.0) 258 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:35.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:35.367779+0000 mon.smithi153 (mon.0) 259 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:35.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:35.369032+0000 mon.smithi153 (mon.0) 260 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:35.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:35 smithi153 bash[14340]: audit 2024-03-12T02:12:35.370297+0000 mon.smithi153 (mon.0) 261 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:12:36.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: cephadm 2024-03-12T02:12:35.372021+0000 mgr.smithi153.asomxr (mgr.14195) 28 : cephadm 1 Updating smithi153:/etc/ceph/ceph.conf 2024-03-12T02:12:36.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: cephadm 2024-03-12T02:12:35.372349+0000 mgr.smithi153.asomxr (mgr.14195) 29 : cephadm 1 Updating smithi163:/etc/ceph/ceph.conf 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: cluster 2024-03-12T02:12:35.417819+0000 mon.smithi153 (mon.0) 262 : cluster 0 mgrmap e18: smithi153.asomxr(active, since 35s), standbys: smithi163.fvmiae 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: audit 2024-03-12T02:12:35.418024+0000 mon.smithi153 (mon.0) 263 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr metadata", "who": "smithi163.fvmiae", "id": "smithi163.fvmiae"}]: dispatch 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: cephadm 2024-03-12T02:12:35.587560+0000 mgr.smithi153.asomxr (mgr.14195) 30 : cephadm 1 Updating smithi163:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: cephadm 2024-03-12T02:12:35.587933+0000 mgr.smithi153.asomxr (mgr.14195) 31 : cephadm 1 Updating smithi153:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: audit 2024-03-12T02:12:35.762826+0000 mon.smithi153 (mon.0) 264 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: audit 2024-03-12T02:12:35.768072+0000 mon.smithi153 (mon.0) 265 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: audit 2024-03-12T02:12:35.772672+0000 mon.smithi153 (mon.0) 266 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: audit 2024-03-12T02:12:35.778940+0000 mon.smithi153 (mon.0) 267 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: audit 2024-03-12T02:12:35.785684+0000 mon.smithi153 (mon.0) 268 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: cephadm 2024-03-12T02:12:35.856273+0000 mgr.smithi153.asomxr (mgr.14195) 32 : cephadm 1 Reconfiguring mon.smithi153 (unknown last config time)... 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: audit 2024-03-12T02:12:35.856850+0000 mon.smithi153 (mon.0) 269 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: audit 2024-03-12T02:12:35.858203+0000 mon.smithi153 (mon.0) 270 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: audit 2024-03-12T02:12:35.859478+0000 mon.smithi153 (mon.0) 271 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:36.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: cephadm 2024-03-12T02:12:35.860925+0000 mgr.smithi153.asomxr (mgr.14195) 33 : cephadm 1 Reconfiguring daemon mon.smithi153 on smithi153 2024-03-12T02:12:36.737 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:36 smithi163 bash[22347]: audit 2024-03-12T02:12:36.270691+0000 mon.smithi153 (mon.0) 272 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:36.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: cephadm 2024-03-12T02:12:35.372021+0000 mgr.smithi153.asomxr (mgr.14195) 28 : cephadm 1 Updating smithi153:/etc/ceph/ceph.conf 2024-03-12T02:12:36.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: cephadm 2024-03-12T02:12:35.372349+0000 mgr.smithi153.asomxr (mgr.14195) 29 : cephadm 1 Updating smithi163:/etc/ceph/ceph.conf 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: cluster 2024-03-12T02:12:35.417819+0000 mon.smithi153 (mon.0) 262 : cluster 0 mgrmap e18: smithi153.asomxr(active, since 35s), standbys: smithi163.fvmiae 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: audit 2024-03-12T02:12:35.418024+0000 mon.smithi153 (mon.0) 263 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr metadata", "who": "smithi163.fvmiae", "id": "smithi163.fvmiae"}]: dispatch 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: cephadm 2024-03-12T02:12:35.587560+0000 mgr.smithi153.asomxr (mgr.14195) 30 : cephadm 1 Updating smithi163:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: cephadm 2024-03-12T02:12:35.587933+0000 mgr.smithi153.asomxr (mgr.14195) 31 : cephadm 1 Updating smithi153:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: audit 2024-03-12T02:12:35.762826+0000 mon.smithi153 (mon.0) 264 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: audit 2024-03-12T02:12:35.768072+0000 mon.smithi153 (mon.0) 265 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: audit 2024-03-12T02:12:35.772672+0000 mon.smithi153 (mon.0) 266 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: audit 2024-03-12T02:12:35.778940+0000 mon.smithi153 (mon.0) 267 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: audit 2024-03-12T02:12:35.785684+0000 mon.smithi153 (mon.0) 268 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: cephadm 2024-03-12T02:12:35.856273+0000 mgr.smithi153.asomxr (mgr.14195) 32 : cephadm 1 Reconfiguring mon.smithi153 (unknown last config time)... 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: audit 2024-03-12T02:12:35.856850+0000 mon.smithi153 (mon.0) 269 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: audit 2024-03-12T02:12:35.858203+0000 mon.smithi153 (mon.0) 270 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: audit 2024-03-12T02:12:35.859478+0000 mon.smithi153 (mon.0) 271 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: cephadm 2024-03-12T02:12:35.860925+0000 mgr.smithi153.asomxr (mgr.14195) 33 : cephadm 1 Reconfiguring daemon mon.smithi153 on smithi153 2024-03-12T02:12:36.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:36 smithi153 bash[14340]: audit 2024-03-12T02:12:36.270691+0000 mon.smithi153 (mon.0) 272 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:12:37.268 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-12T02:12:37.269 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":2,"fsid":"5cbf0f4e-e015-11ee-95c8-87774f69a715","modified":"2024-03-12T02:12:30.270154Z","created":"2024-03-12T02:09:01.494884Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi163","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:3300","nonce":0},{"type":"v1","addr":"172.21.15.163:6789","nonce":0}]},"addr":"172.21.15.163:6789/0","public_addr":"172.21.15.163:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-12T02:12:37.272 INFO:teuthology.orchestra.run.smithi163.stderr:dumped monmap epoch 2 2024-03-12T02:12:37.976 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-12T02:12:37.976 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph config generate-minimal-conf 2024-03-12T02:12:38.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:37 smithi153 bash[14340]: cluster 2024-03-12T02:12:36.313864+0000 mgr.smithi153.asomxr (mgr.14195) 34 : cluster 0 pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:38.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:37 smithi153 bash[14340]: audit 2024-03-12T02:12:37.266590+0000 mon.smithi153 (mon.0) 273 : audit 0 from='client.? 172.21.15.163:0/2011035749' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T02:12:38.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:37 smithi163 bash[22347]: cluster 2024-03-12T02:12:36.313864+0000 mgr.smithi153.asomxr (mgr.14195) 34 : cluster 0 pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:38.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:37 smithi163 bash[22347]: audit 2024-03-12T02:12:37.266590+0000 mon.smithi153 (mon.0) 273 : audit 0 from='client.? 172.21.15.163:0/2011035749' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-12T02:12:39.193 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:38 smithi153 bash[14340]: audit 2024-03-12T02:12:37.833859+0000 mon.smithi153 (mon.0) 274 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:39.193 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:38 smithi153 bash[14340]: audit 2024-03-12T02:12:37.842965+0000 mon.smithi153 (mon.0) 275 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:39.193 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:38 smithi153 bash[14340]: audit 2024-03-12T02:12:37.880451+0000 mon.smithi153 (mon.0) 276 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi153.asomxr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T02:12:39.193 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:38 smithi153 bash[14340]: audit 2024-03-12T02:12:37.882443+0000 mon.smithi153 (mon.0) 277 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T02:12:39.193 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:38 smithi153 bash[14340]: audit 2024-03-12T02:12:37.883953+0000 mon.smithi153 (mon.0) 278 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:39.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:38 smithi163 bash[22347]: audit 2024-03-12T02:12:37.833859+0000 mon.smithi153 (mon.0) 274 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:39.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:38 smithi163 bash[22347]: audit 2024-03-12T02:12:37.842965+0000 mon.smithi153 (mon.0) 275 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:39.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:38 smithi163 bash[22347]: audit 2024-03-12T02:12:37.880451+0000 mon.smithi153 (mon.0) 276 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi153.asomxr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T02:12:39.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:38 smithi163 bash[22347]: audit 2024-03-12T02:12:37.882443+0000 mon.smithi153 (mon.0) 277 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T02:12:39.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:38 smithi163 bash[22347]: audit 2024-03-12T02:12:37.883953+0000 mon.smithi153 (mon.0) 278 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:40.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:39 smithi163 bash[22347]: cephadm 2024-03-12T02:12:37.879703+0000 mgr.smithi153.asomxr (mgr.14195) 35 : cephadm 1 Reconfiguring mgr.smithi153.asomxr (unknown last config time)... 2024-03-12T02:12:40.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:39 smithi163 bash[22347]: cephadm 2024-03-12T02:12:37.885752+0000 mgr.smithi153.asomxr (mgr.14195) 36 : cephadm 1 Reconfiguring daemon mgr.smithi153.asomxr on smithi153 2024-03-12T02:12:40.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:39 smithi163 bash[22347]: cluster 2024-03-12T02:12:38.314242+0000 mgr.smithi153.asomxr (mgr.14195) 37 : cluster 0 pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:40.256 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:39 smithi153 bash[14340]: cephadm 2024-03-12T02:12:37.879703+0000 mgr.smithi153.asomxr (mgr.14195) 35 : cephadm 1 Reconfiguring mgr.smithi153.asomxr (unknown last config time)... 2024-03-12T02:12:40.256 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:39 smithi153 bash[14340]: cephadm 2024-03-12T02:12:37.885752+0000 mgr.smithi153.asomxr (mgr.14195) 36 : cephadm 1 Reconfiguring daemon mgr.smithi153.asomxr on smithi153 2024-03-12T02:12:40.256 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:39 smithi153 bash[14340]: cluster 2024-03-12T02:12:38.314242+0000 mgr.smithi153.asomxr (mgr.14195) 37 : cluster 0 pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:41.112 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:40 smithi153 bash[14340]: audit 2024-03-12T02:12:39.911846+0000 mon.smithi153 (mon.0) 279 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:41.112 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:40 smithi153 bash[14340]: audit 2024-03-12T02:12:39.919057+0000 mon.smithi153 (mon.0) 280 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:41.112 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:40 smithi153 bash[14340]: cephadm 2024-03-12T02:12:39.955008+0000 mgr.smithi153.asomxr (mgr.14195) 38 : cephadm 1 Reconfiguring ceph-exporter.smithi153 (monmap changed)... 2024-03-12T02:12:41.112 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:40 smithi153 bash[14340]: audit 2024-03-12T02:12:39.955730+0000 mon.smithi153 (mon.0) 281 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-12T02:12:41.112 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:40 smithi153 bash[14340]: audit 2024-03-12T02:12:39.957803+0000 mon.smithi153 (mon.0) 282 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:41.113 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:40 smithi153 bash[14340]: cephadm 2024-03-12T02:12:39.959634+0000 mgr.smithi153.asomxr (mgr.14195) 39 : cephadm 1 Reconfiguring daemon ceph-exporter.smithi153 on smithi153 2024-03-12T02:12:41.113 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:40 smithi153 bash[14340]: cluster 2024-03-12T02:12:40.314662+0000 mgr.smithi153.asomxr (mgr.14195) 40 : cluster 0 pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:41.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:40 smithi163 bash[22347]: audit 2024-03-12T02:12:39.911846+0000 mon.smithi153 (mon.0) 279 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:41.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:40 smithi163 bash[22347]: audit 2024-03-12T02:12:39.919057+0000 mon.smithi153 (mon.0) 280 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:41.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:40 smithi163 bash[22347]: cephadm 2024-03-12T02:12:39.955008+0000 mgr.smithi153.asomxr (mgr.14195) 38 : cephadm 1 Reconfiguring ceph-exporter.smithi153 (monmap changed)... 2024-03-12T02:12:41.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:40 smithi163 bash[22347]: audit 2024-03-12T02:12:39.955730+0000 mon.smithi153 (mon.0) 281 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-12T02:12:41.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:40 smithi163 bash[22347]: audit 2024-03-12T02:12:39.957803+0000 mon.smithi153 (mon.0) 282 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:41.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:40 smithi163 bash[22347]: cephadm 2024-03-12T02:12:39.959634+0000 mgr.smithi153.asomxr (mgr.14195) 39 : cephadm 1 Reconfiguring daemon ceph-exporter.smithi153 on smithi153 2024-03-12T02:12:41.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:40 smithi163 bash[22347]: cluster 2024-03-12T02:12:40.314662+0000 mgr.smithi153.asomxr (mgr.14195) 40 : cluster 0 pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:42.676 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:12:43.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:42 smithi163 bash[22347]: audit 2024-03-12T02:12:41.972342+0000 mon.smithi153 (mon.0) 283 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:43.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:42 smithi163 bash[22347]: audit 2024-03-12T02:12:41.979474+0000 mon.smithi153 (mon.0) 284 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:43.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:42 smithi163 bash[22347]: cephadm 2024-03-12T02:12:42.030312+0000 mgr.smithi153.asomxr (mgr.14195) 41 : cephadm 1 Reconfiguring crash.smithi153 (monmap changed)... 2024-03-12T02:12:43.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:42 smithi163 bash[22347]: audit 2024-03-12T02:12:42.031005+0000 mon.smithi153 (mon.0) 285 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-12T02:12:43.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:42 smithi163 bash[22347]: audit 2024-03-12T02:12:42.032980+0000 mon.smithi153 (mon.0) 286 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:43.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:42 smithi163 bash[22347]: cephadm 2024-03-12T02:12:42.034912+0000 mgr.smithi153.asomxr (mgr.14195) 42 : cephadm 1 Reconfiguring daemon crash.smithi153 on smithi153 2024-03-12T02:12:43.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:42 smithi163 bash[22347]: cluster 2024-03-12T02:12:42.315093+0000 mgr.smithi153.asomxr (mgr.14195) 43 : cluster 0 pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:43.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:42 smithi153 bash[14340]: audit 2024-03-12T02:12:41.972342+0000 mon.smithi153 (mon.0) 283 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:43.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:42 smithi153 bash[14340]: audit 2024-03-12T02:12:41.979474+0000 mon.smithi153 (mon.0) 284 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:43.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:42 smithi153 bash[14340]: cephadm 2024-03-12T02:12:42.030312+0000 mgr.smithi153.asomxr (mgr.14195) 41 : cephadm 1 Reconfiguring crash.smithi153 (monmap changed)... 2024-03-12T02:12:43.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:42 smithi153 bash[14340]: audit 2024-03-12T02:12:42.031005+0000 mon.smithi153 (mon.0) 285 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-12T02:12:43.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:42 smithi153 bash[14340]: audit 2024-03-12T02:12:42.032980+0000 mon.smithi153 (mon.0) 286 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:43.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:42 smithi153 bash[14340]: cephadm 2024-03-12T02:12:42.034912+0000 mgr.smithi153.asomxr (mgr.14195) 42 : cephadm 1 Reconfiguring daemon crash.smithi153 on smithi153 2024-03-12T02:12:43.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:42 smithi153 bash[14340]: cluster 2024-03-12T02:12:42.315093+0000 mgr.smithi153.asomxr (mgr.14195) 43 : cluster 0 pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:45.695 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:45 smithi163 bash[22347]: cluster 2024-03-12T02:12:44.315545+0000 mgr.smithi153.asomxr (mgr.14195) 44 : cluster 0 pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:45.695 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:45 smithi163 bash[22347]: audit 2024-03-12T02:12:45.224663+0000 mon.smithi153 (mon.0) 287 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:45.695 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:45 smithi163 bash[22347]: audit 2024-03-12T02:12:45.231410+0000 mon.smithi153 (mon.0) 288 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:45.695 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:45 smithi163 bash[22347]: audit 2024-03-12T02:12:45.367923+0000 mon.smithi153 (mon.0) 289 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:12:45.713 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:45 smithi153 bash[14340]: cluster 2024-03-12T02:12:44.315545+0000 mgr.smithi153.asomxr (mgr.14195) 44 : cluster 0 pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:45.713 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:45 smithi153 bash[14340]: audit 2024-03-12T02:12:45.224663+0000 mon.smithi153 (mon.0) 287 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:45.713 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:45 smithi153 bash[14340]: audit 2024-03-12T02:12:45.231410+0000 mon.smithi153 (mon.0) 288 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:45.713 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:45 smithi153 bash[14340]: audit 2024-03-12T02:12:45.367923+0000 mon.smithi153 (mon.0) 289 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:12:46.600 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:46 smithi153 bash[14340]: cephadm 2024-03-12T02:12:45.256242+0000 mgr.smithi153.asomxr (mgr.14195) 45 : cephadm 1 Reconfiguring alertmanager.smithi153 (dependencies changed)... 2024-03-12T02:12:46.601 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:46 smithi153 bash[14340]: cephadm 2024-03-12T02:12:45.279861+0000 mgr.smithi153.asomxr (mgr.14195) 46 : cephadm 1 Reconfiguring daemon alertmanager.smithi153 on smithi153 2024-03-12T02:12:46.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:46 smithi163 bash[22347]: cephadm 2024-03-12T02:12:45.256242+0000 mgr.smithi153.asomxr (mgr.14195) 45 : cephadm 1 Reconfiguring alertmanager.smithi153 (dependencies changed)... 2024-03-12T02:12:46.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:46 smithi163 bash[22347]: cephadm 2024-03-12T02:12:45.279861+0000 mgr.smithi153.asomxr (mgr.14195) 46 : cephadm 1 Reconfiguring daemon alertmanager.smithi153 on smithi153 2024-03-12T02:12:47.461 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:47 smithi153 bash[14340]: cluster 2024-03-12T02:12:46.315868+0000 mgr.smithi153.asomxr (mgr.14195) 47 : cluster 0 pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:47.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:47 smithi163 bash[22347]: cluster 2024-03-12T02:12:46.315868+0000 mgr.smithi153.asomxr (mgr.14195) 47 : cluster 0 pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:47.585 INFO:teuthology.orchestra.run.smithi153.stdout:# minimal ceph.conf for 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2024-03-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T02:12:47.586 INFO:teuthology.orchestra.run.smithi153.stdout: mon_host = [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] 2024-03-12T02:12:48.611 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-12T02:12:48.611 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:12:48.611 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.conf 2024-03-12T02:12:48.699 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:12:48.699 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T02:12:48.767 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T02:12:48.767 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.conf 2024-03-12T02:12:48.785 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T02:12:48.785 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-12T02:12:48.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:48 smithi153 bash[14340]: audit 2024-03-12T02:12:47.584021+0000 mon.smithi153 (mon.0) 290 : audit 0 from='client.? 172.21.15.153:0/1560422868' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:48.845 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:48 smithi163 bash[22347]: audit 2024-03-12T02:12:47.584021+0000 mon.smithi153 (mon.0) 290 : audit 0 from='client.? 172.21.15.153:0/1560422868' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:12:48.850 INFO:tasks.cephadm:Deploying OSDs... 2024-03-12T02:12:48.850 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:12:48.850 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T02:12:48.857 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-03-12T02:12:48.857 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/nvme1n1 2024-03-12T02:12:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/nvme1n1 2024-03-12T02:12:48.905 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-03-12T02:12:48.905 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 522 Links: 1 Device type: 103,2 2024-03-12T02:12:48.905 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-03-12T02:12:48.906 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-12 02:10:44.491371666 +0000 2024-03-12T02:12:48.906 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-12 02:08:03.079065878 +0000 2024-03-12T02:12:48.906 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-12 02:08:03.079065878 +0000 2024-03-12T02:12:48.906 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2024-03-12T02:12:48.906 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-03-12T02:12:48.961 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-12T02:12:48.961 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-12T02:12:48.961 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000478646 s, 1.1 MB/s 2024-03-12T02:12:48.962 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-03-12T02:12:49.015 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/nvme2n1 2024-03-12T02:12:49.064 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/nvme2n1 2024-03-12T02:12:49.065 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-03-12T02:12:49.065 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 530 Links: 1 Device type: 103,4 2024-03-12T02:12:49.065 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-03-12T02:12:49.065 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-12 02:10:44.511371205 +0000 2024-03-12T02:12:49.065 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-12 02:08:03.191066320 +0000 2024-03-12T02:12:49.065 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-12 02:08:03.191066320 +0000 2024-03-12T02:12:49.065 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2024-03-12T02:12:49.065 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-03-12T02:12:49.120 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-12T02:12:49.120 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-12T02:12:49.120 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000497361 s, 1.0 MB/s 2024-03-12T02:12:49.121 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-03-12T02:12:49.171 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/nvme3n1 2024-03-12T02:12:49.218 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/nvme3n1 2024-03-12T02:12:49.218 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-03-12T02:12:49.219 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 538 Links: 1 Device type: 103,6 2024-03-12T02:12:49.219 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-03-12T02:12:49.219 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-12 02:10:44.531370745 +0000 2024-03-12T02:12:49.219 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-12 02:08:03.303066763 +0000 2024-03-12T02:12:49.219 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-12 02:08:03.303066763 +0000 2024-03-12T02:12:49.219 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2024-03-12T02:12:49.219 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-03-12T02:12:49.271 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-12T02:12:49.271 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-12T02:12:49.271 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000490315 s, 1.0 MB/s 2024-03-12T02:12:49.272 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-03-12T02:12:49.327 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/nvme4n1 2024-03-12T02:12:49.377 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/nvme4n1 2024-03-12T02:12:49.377 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-03-12T02:12:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 546 Links: 1 Device type: 103,8 2024-03-12T02:12:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-03-12T02:12:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-12 02:10:44.551370284 +0000 2024-03-12T02:12:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-12 02:08:03.411067190 +0000 2024-03-12T02:12:49.377 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-12 02:08:03.411067190 +0000 2024-03-12T02:12:49.377 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2024-03-12T02:12:49.378 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-03-12T02:12:49.434 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-12T02:12:49.434 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-12T02:12:49.434 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000431678 s, 1.2 MB/s 2024-03-12T02:12:49.435 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-03-12T02:12:49.487 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T02:12:49.487 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2024-03-12T02:12:49.495 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-03-12T02:12:49.496 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/nvme1n1 2024-03-12T02:12:49.544 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/nvme1n1 2024-03-12T02:12:49.544 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-03-12T02:12:49.544 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 522 Links: 1 Device type: 103,2 2024-03-12T02:12:49.544 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-03-12T02:12:49.545 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-12 02:12:09.262865369 +0000 2024-03-12T02:12:49.545 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-12 02:08:04.418724477 +0000 2024-03-12T02:12:49.545 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-12 02:08:04.418724477 +0000 2024-03-12T02:12:49.545 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-03-12T02:12:49.545 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-03-12T02:12:49.598 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-12T02:12:49.599 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-12T02:12:49.599 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000475061 s, 1.1 MB/s 2024-03-12T02:12:49.600 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-03-12T02:12:49.650 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/nvme2n1 2024-03-12T02:12:49.704 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/nvme2n1 2024-03-12T02:12:49.704 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-03-12T02:12:49.704 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 530 Links: 1 Device type: 103,4 2024-03-12T02:12:49.704 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-03-12T02:12:49.705 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-12 02:12:09.282864898 +0000 2024-03-12T02:12:49.705 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-12 02:08:04.530665894 +0000 2024-03-12T02:12:49.705 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-12 02:08:04.530665894 +0000 2024-03-12T02:12:49.705 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-03-12T02:12:49.705 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-03-12T02:12:49.762 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-12T02:12:49.762 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-12T02:12:49.762 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000458464 s, 1.1 MB/s 2024-03-12T02:12:49.763 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-03-12T02:12:49.818 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/nvme3n1 2024-03-12T02:12:49.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:49 smithi153 bash[14340]: cluster 2024-03-12T02:12:48.316278+0000 mgr.smithi153.asomxr (mgr.14195) 48 : cluster 0 pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:49.872 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/nvme3n1 2024-03-12T02:12:49.872 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-03-12T02:12:49.872 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 538 Links: 1 Device type: 103,6 2024-03-12T02:12:49.872 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-03-12T02:12:49.872 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-12 02:12:09.302864429 +0000 2024-03-12T02:12:49.872 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-12 02:08:04.650603128 +0000 2024-03-12T02:12:49.872 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-12 02:08:04.650603128 +0000 2024-03-12T02:12:49.872 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-03-12T02:12:49.873 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-03-12T02:12:49.925 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:49 smithi163 bash[22347]: cluster 2024-03-12T02:12:48.316278+0000 mgr.smithi153.asomxr (mgr.14195) 48 : cluster 0 pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-12T02:12:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-12T02:12:49.928 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000485675 s, 1.1 MB/s 2024-03-12T02:12:49.928 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-03-12T02:12:49.978 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/nvme4n1 2024-03-12T02:12:50.032 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/nvme4n1 2024-03-12T02:12:50.032 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-03-12T02:12:50.032 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 546 Links: 1 Device type: 103,8 2024-03-12T02:12:50.033 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-03-12T02:12:50.033 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-12 02:12:09.322863960 +0000 2024-03-12T02:12:50.033 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-12 02:08:04.762544544 +0000 2024-03-12T02:12:50.033 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-12 02:08:04.762544544 +0000 2024-03-12T02:12:50.033 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2024-03-12T02:12:50.033 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-03-12T02:12:50.087 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-12T02:12:50.087 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-12T02:12:50.088 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.00046076 s, 1.1 MB/s 2024-03-12T02:12:50.089 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-03-12T02:12:50.142 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch apply osd --all-available-devices 2024-03-12T02:12:51.763 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:51 smithi153 bash[14340]: cluster 2024-03-12T02:12:50.316761+0000 mgr.smithi153.asomxr (mgr.14195) 49 : cluster 0 pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:51.763 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:51 smithi153 bash[14340]: audit 2024-03-12T02:12:51.442950+0000 mon.smithi153 (mon.0) 291 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:51.763 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:51 smithi153 bash[14340]: audit 2024-03-12T02:12:51.451957+0000 mon.smithi153 (mon.0) 292 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:51.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:51 smithi163 bash[22347]: cluster 2024-03-12T02:12:50.316761+0000 mgr.smithi153.asomxr (mgr.14195) 49 : cluster 0 pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:51 smithi163 bash[22347]: audit 2024-03-12T02:12:51.442950+0000 mon.smithi153 (mon.0) 291 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:51 smithi163 bash[22347]: audit 2024-03-12T02:12:51.451957+0000 mon.smithi153 (mon.0) 292 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:52.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:52 smithi153 bash[14340]: cephadm 2024-03-12T02:12:51.459416+0000 mgr.smithi153.asomxr (mgr.14195) 50 : cephadm 1 Reconfiguring grafana.smithi153 (dependencies changed)... 2024-03-12T02:12:52.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:52 smithi153 bash[14340]: cephadm 2024-03-12T02:12:51.482262+0000 mgr.smithi153.asomxr (mgr.14195) 51 : cephadm 1 Reconfiguring daemon grafana.smithi153 on smithi153 2024-03-12T02:12:52.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:52 smithi163 bash[22347]: cephadm 2024-03-12T02:12:51.459416+0000 mgr.smithi153.asomxr (mgr.14195) 50 : cephadm 1 Reconfiguring grafana.smithi153 (dependencies changed)... 2024-03-12T02:12:52.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:52 smithi163 bash[22347]: cephadm 2024-03-12T02:12:51.482262+0000 mgr.smithi153.asomxr (mgr.14195) 51 : cephadm 1 Reconfiguring daemon grafana.smithi153 on smithi153 2024-03-12T02:12:53.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:53 smithi153 bash[14340]: cluster 2024-03-12T02:12:52.317223+0000 mgr.smithi153.asomxr (mgr.14195) 52 : cluster 0 pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:53.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:53 smithi163 bash[22347]: cluster 2024-03-12T02:12:52.317223+0000 mgr.smithi153.asomxr (mgr.14195) 52 : cluster 0 pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:54.889 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi163/config 2024-03-12T02:12:55.864 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:55 smithi163 bash[22347]: cluster 2024-03-12T02:12:54.317652+0000 mgr.smithi153.asomxr (mgr.14195) 53 : cluster 0 pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:55.889 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:55 smithi153 bash[14340]: cluster 2024-03-12T02:12:54.317652+0000 mgr.smithi153.asomxr (mgr.14195) 53 : cluster 0 pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:57.779 INFO:teuthology.orchestra.run.smithi163.stdout:Scheduled osd.all-available-devices update... 2024-03-12T02:12:57.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:57 smithi153 bash[14340]: cluster 2024-03-12T02:12:56.318065+0000 mgr.smithi153.asomxr (mgr.14195) 54 : cluster 0 pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:57.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:57 smithi153 bash[14340]: audit 2024-03-12T02:12:56.628733+0000 mon.smithi153 (mon.0) 293 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:57.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:57 smithi153 bash[14340]: audit 2024-03-12T02:12:56.636045+0000 mon.smithi153 (mon.0) 294 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:57.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:57 smithi153 bash[14340]: cephadm 2024-03-12T02:12:56.659136+0000 mgr.smithi153.asomxr (mgr.14195) 55 : cephadm 1 Reconfiguring prometheus.smithi153 (dependencies changed)... 2024-03-12T02:12:57.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:57 smithi163 bash[22347]: cluster 2024-03-12T02:12:56.318065+0000 mgr.smithi153.asomxr (mgr.14195) 54 : cluster 0 pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:12:57.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:57 smithi163 bash[22347]: audit 2024-03-12T02:12:56.628733+0000 mon.smithi153 (mon.0) 293 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:57.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:57 smithi163 bash[22347]: audit 2024-03-12T02:12:56.636045+0000 mon.smithi153 (mon.0) 294 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:57.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:57 smithi163 bash[22347]: cephadm 2024-03-12T02:12:56.659136+0000 mgr.smithi153.asomxr (mgr.14195) 55 : cephadm 1 Reconfiguring prometheus.smithi153 (dependencies changed)... 2024-03-12T02:12:58.449 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-03-12T02:12:58.450 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T02:12:58.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:58 smithi153 bash[14340]: cephadm 2024-03-12T02:12:56.903274+0000 mgr.smithi153.asomxr (mgr.14195) 56 : cephadm 1 Reconfiguring daemon prometheus.smithi153 on smithi153 2024-03-12T02:12:58.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:58 smithi153 bash[14340]: audit 2024-03-12T02:12:57.766479+0000 mgr.smithi153.asomxr (mgr.14195) 57 : audit 0 from='client.24099 -' entity='client.admin' cmd=[{"prefix": "orch apply osd", "all_available_devices": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:12:58.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:58 smithi153 bash[14340]: cephadm 2024-03-12T02:12:57.770103+0000 mgr.smithi153.asomxr (mgr.14195) 58 : cephadm 1 Marking host: smithi153 for OSDSpec preview refresh. 2024-03-12T02:12:58.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:58 smithi153 bash[14340]: cephadm 2024-03-12T02:12:57.770182+0000 mgr.smithi153.asomxr (mgr.14195) 59 : cephadm 1 Marking host: smithi163 for OSDSpec preview refresh. 2024-03-12T02:12:58.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:58 smithi153 bash[14340]: cephadm 2024-03-12T02:12:57.770559+0000 mgr.smithi153.asomxr (mgr.14195) 60 : cephadm 1 Saving service osd.all-available-devices spec with placement * 2024-03-12T02:12:58.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:58 smithi153 bash[14340]: audit 2024-03-12T02:12:57.776683+0000 mon.smithi153 (mon.0) 295 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:59.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:58 smithi163 bash[22347]: cephadm 2024-03-12T02:12:56.903274+0000 mgr.smithi153.asomxr (mgr.14195) 56 : cephadm 1 Reconfiguring daemon prometheus.smithi153 on smithi153 2024-03-12T02:12:59.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:58 smithi163 bash[22347]: audit 2024-03-12T02:12:57.766479+0000 mgr.smithi153.asomxr (mgr.14195) 57 : audit 0 from='client.24099 -' entity='client.admin' cmd=[{"prefix": "orch apply osd", "all_available_devices": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:12:59.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:58 smithi163 bash[22347]: cephadm 2024-03-12T02:12:57.770103+0000 mgr.smithi153.asomxr (mgr.14195) 58 : cephadm 1 Marking host: smithi153 for OSDSpec preview refresh. 2024-03-12T02:12:59.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:58 smithi163 bash[22347]: cephadm 2024-03-12T02:12:57.770182+0000 mgr.smithi153.asomxr (mgr.14195) 59 : cephadm 1 Marking host: smithi163 for OSDSpec preview refresh. 2024-03-12T02:12:59.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:58 smithi163 bash[22347]: cephadm 2024-03-12T02:12:57.770559+0000 mgr.smithi153.asomxr (mgr.14195) 60 : cephadm 1 Saving service osd.all-available-devices spec with placement * 2024-03-12T02:12:59.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:58 smithi163 bash[22347]: audit 2024-03-12T02:12:57.776683+0000 mon.smithi153 (mon.0) 295 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:12:59.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:12:59 smithi153 bash[14340]: cluster 2024-03-12T02:12:58.318344+0000 mgr.smithi153.asomxr (mgr.14195) 61 : cluster 0 pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:00.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:12:59 smithi163 bash[22347]: cluster 2024-03-12T02:12:58.318344+0000 mgr.smithi153.asomxr (mgr.14195) 61 : cluster 0 pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:01.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:00 smithi153 bash[14340]: audit 2024-03-12T02:13:00.368393+0000 mon.smithi153 (mon.0) 296 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:13:01.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:00 smithi163 bash[22347]: audit 2024-03-12T02:13:00.368393+0000 mon.smithi153 (mon.0) 296 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:13:02.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:01 smithi153 bash[14340]: cluster 2024-03-12T02:13:00.318737+0000 mgr.smithi153.asomxr (mgr.14195) 62 : cluster 0 pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:02.227 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:01 smithi163 bash[22347]: cluster 2024-03-12T02:13:00.318737+0000 mgr.smithi153.asomxr (mgr.14195) 62 : cluster 0 pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:03.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:03 smithi163 bash[22347]: cluster 2024-03-12T02:13:02.319206+0000 mgr.smithi153.asomxr (mgr.14195) 63 : cluster 0 pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:03.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:03 smithi163 bash[22347]: audit 2024-03-12T02:13:02.404333+0000 mon.smithi153 (mon.0) 297 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:03.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:03 smithi163 bash[22347]: audit 2024-03-12T02:13:02.412846+0000 mon.smithi153 (mon.0) 298 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:03.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:03 smithi163 bash[22347]: cephadm 2024-03-12T02:13:02.445035+0000 mgr.smithi153.asomxr (mgr.14195) 64 : cephadm 1 Reconfiguring ceph-exporter.smithi163 (monmap changed)... 2024-03-12T02:13:03.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:03 smithi163 bash[22347]: audit 2024-03-12T02:13:02.445873+0000 mon.smithi153 (mon.0) 299 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi163", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-12T02:13:03.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:03 smithi163 bash[22347]: audit 2024-03-12T02:13:02.447684+0000 mon.smithi153 (mon.0) 300 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:03.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:03 smithi163 bash[22347]: cephadm 2024-03-12T02:13:02.448995+0000 mgr.smithi153.asomxr (mgr.14195) 65 : cephadm 1 Reconfiguring daemon ceph-exporter.smithi163 on smithi163 2024-03-12T02:13:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:03 smithi153 bash[14340]: cluster 2024-03-12T02:13:02.319206+0000 mgr.smithi153.asomxr (mgr.14195) 63 : cluster 0 pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:03 smithi153 bash[14340]: audit 2024-03-12T02:13:02.404333+0000 mon.smithi153 (mon.0) 297 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:03 smithi153 bash[14340]: audit 2024-03-12T02:13:02.412846+0000 mon.smithi153 (mon.0) 298 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:03 smithi153 bash[14340]: cephadm 2024-03-12T02:13:02.445035+0000 mgr.smithi153.asomxr (mgr.14195) 64 : cephadm 1 Reconfiguring ceph-exporter.smithi163 (monmap changed)... 2024-03-12T02:13:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:03 smithi153 bash[14340]: audit 2024-03-12T02:13:02.445873+0000 mon.smithi153 (mon.0) 299 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi163", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-12T02:13:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:03 smithi153 bash[14340]: audit 2024-03-12T02:13:02.447684+0000 mon.smithi153 (mon.0) 300 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:03 smithi153 bash[14340]: cephadm 2024-03-12T02:13:02.448995+0000 mgr.smithi153.asomxr (mgr.14195) 65 : cephadm 1 Reconfiguring daemon ceph-exporter.smithi163 on smithi163 2024-03-12T02:13:05.424 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:05 smithi163 bash[22347]: audit 2024-03-12T02:13:04.172402+0000 mon.smithi153 (mon.0) 301 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:05.425 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:05 smithi163 bash[22347]: audit 2024-03-12T02:13:04.178430+0000 mon.smithi153 (mon.0) 302 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:05.425 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:05 smithi163 bash[22347]: cephadm 2024-03-12T02:13:04.218578+0000 mgr.smithi153.asomxr (mgr.14195) 66 : cephadm 1 Reconfiguring crash.smithi163 (monmap changed)... 2024-03-12T02:13:05.425 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:05 smithi163 bash[22347]: audit 2024-03-12T02:13:04.219227+0000 mon.smithi153 (mon.0) 303 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-12T02:13:05.425 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:05 smithi163 bash[22347]: audit 2024-03-12T02:13:04.221318+0000 mon.smithi153 (mon.0) 304 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:05.425 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:05 smithi163 bash[22347]: cephadm 2024-03-12T02:13:04.223205+0000 mgr.smithi153.asomxr (mgr.14195) 67 : cephadm 1 Reconfiguring daemon crash.smithi163 on smithi163 2024-03-12T02:13:05.425 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:05 smithi163 bash[22347]: cluster 2024-03-12T02:13:04.319647+0000 mgr.smithi153.asomxr (mgr.14195) 68 : cluster 0 pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:05.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:05 smithi153 bash[14340]: audit 2024-03-12T02:13:04.172402+0000 mon.smithi153 (mon.0) 301 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:05.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:05 smithi153 bash[14340]: audit 2024-03-12T02:13:04.178430+0000 mon.smithi153 (mon.0) 302 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:05.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:05 smithi153 bash[14340]: cephadm 2024-03-12T02:13:04.218578+0000 mgr.smithi153.asomxr (mgr.14195) 66 : cephadm 1 Reconfiguring crash.smithi163 (monmap changed)... 2024-03-12T02:13:05.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:05 smithi153 bash[14340]: audit 2024-03-12T02:13:04.219227+0000 mon.smithi153 (mon.0) 303 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi163", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-12T02:13:05.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:05 smithi153 bash[14340]: audit 2024-03-12T02:13:04.221318+0000 mon.smithi153 (mon.0) 304 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:05.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:05 smithi153 bash[14340]: cephadm 2024-03-12T02:13:04.223205+0000 mgr.smithi153.asomxr (mgr.14195) 67 : cephadm 1 Reconfiguring daemon crash.smithi163 on smithi163 2024-03-12T02:13:05.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:05 smithi153 bash[14340]: cluster 2024-03-12T02:13:04.319647+0000 mgr.smithi153.asomxr (mgr.14195) 68 : cluster 0 pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:05.874 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:13:07.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:06 smithi163 bash[22347]: audit 2024-03-12T02:13:05.999203+0000 mon.smithi153 (mon.0) 305 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:07.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:06 smithi163 bash[22347]: audit 2024-03-12T02:13:06.006615+0000 mon.smithi153 (mon.0) 306 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:07.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:06 smithi163 bash[22347]: cephadm 2024-03-12T02:13:06.042569+0000 mgr.smithi153.asomxr (mgr.14195) 69 : cephadm 1 Reconfiguring mgr.smithi163.fvmiae (monmap changed)... 2024-03-12T02:13:07.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:06 smithi163 bash[22347]: audit 2024-03-12T02:13:06.043149+0000 mon.smithi153 (mon.0) 307 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.fvmiae", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T02:13:07.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:06 smithi163 bash[22347]: audit 2024-03-12T02:13:06.044758+0000 mon.smithi153 (mon.0) 308 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T02:13:07.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:06 smithi163 bash[22347]: audit 2024-03-12T02:13:06.045952+0000 mon.smithi153 (mon.0) 309 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:07.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:06 smithi163 bash[22347]: cephadm 2024-03-12T02:13:06.047354+0000 mgr.smithi153.asomxr (mgr.14195) 70 : cephadm 1 Reconfiguring daemon mgr.smithi163.fvmiae on smithi163 2024-03-12T02:13:07.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:06 smithi163 bash[22347]: cluster 2024-03-12T02:13:06.320020+0000 mgr.smithi153.asomxr (mgr.14195) 71 : cluster 0 pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:07.253 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:06 smithi153 bash[14340]: audit 2024-03-12T02:13:05.999203+0000 mon.smithi153 (mon.0) 305 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:07.253 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:06 smithi153 bash[14340]: audit 2024-03-12T02:13:06.006615+0000 mon.smithi153 (mon.0) 306 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:07.253 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:06 smithi153 bash[14340]: cephadm 2024-03-12T02:13:06.042569+0000 mgr.smithi153.asomxr (mgr.14195) 69 : cephadm 1 Reconfiguring mgr.smithi163.fvmiae (monmap changed)... 2024-03-12T02:13:07.253 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:06 smithi153 bash[14340]: audit 2024-03-12T02:13:06.043149+0000 mon.smithi153 (mon.0) 307 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi163.fvmiae", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-12T02:13:07.253 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:06 smithi153 bash[14340]: audit 2024-03-12T02:13:06.044758+0000 mon.smithi153 (mon.0) 308 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-12T02:13:07.253 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:06 smithi153 bash[14340]: audit 2024-03-12T02:13:06.045952+0000 mon.smithi153 (mon.0) 309 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:07.253 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:06 smithi153 bash[14340]: cephadm 2024-03-12T02:13:06.047354+0000 mgr.smithi153.asomxr (mgr.14195) 70 : cephadm 1 Reconfiguring daemon mgr.smithi163.fvmiae on smithi163 2024-03-12T02:13:07.254 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:06 smithi153 bash[14340]: cluster 2024-03-12T02:13:06.320020+0000 mgr.smithi153.asomxr (mgr.14195) 71 : cluster 0 pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:08.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:08 smithi163 bash[22347]: audit 2024-03-12T02:13:07.721637+0000 mon.smithi153 (mon.0) 310 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:08.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:08 smithi163 bash[22347]: audit 2024-03-12T02:13:07.728613+0000 mon.smithi153 (mon.0) 311 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:08.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:08 smithi163 bash[22347]: cephadm 2024-03-12T02:13:07.765740+0000 mgr.smithi153.asomxr (mgr.14195) 72 : cephadm 1 Reconfiguring mon.smithi163 (monmap changed)... 2024-03-12T02:13:08.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:08 smithi163 bash[22347]: audit 2024-03-12T02:13:07.766287+0000 mon.smithi153 (mon.0) 312 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T02:13:08.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:08 smithi163 bash[22347]: audit 2024-03-12T02:13:07.767713+0000 mon.smithi153 (mon.0) 313 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T02:13:08.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:08 smithi163 bash[22347]: audit 2024-03-12T02:13:07.769109+0000 mon.smithi153 (mon.0) 314 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:08.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:08 smithi163 bash[22347]: cephadm 2024-03-12T02:13:07.770761+0000 mgr.smithi153.asomxr (mgr.14195) 73 : cephadm 1 Reconfiguring daemon mon.smithi163 on smithi163 2024-03-12T02:13:09.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:08 smithi153 bash[14340]: audit 2024-03-12T02:13:07.721637+0000 mon.smithi153 (mon.0) 310 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:08 smithi153 bash[14340]: audit 2024-03-12T02:13:07.728613+0000 mon.smithi153 (mon.0) 311 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:08 smithi153 bash[14340]: cephadm 2024-03-12T02:13:07.765740+0000 mgr.smithi153.asomxr (mgr.14195) 72 : cephadm 1 Reconfiguring mon.smithi163 (monmap changed)... 2024-03-12T02:13:09.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:08 smithi153 bash[14340]: audit 2024-03-12T02:13:07.766287+0000 mon.smithi153 (mon.0) 312 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-12T02:13:09.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:08 smithi153 bash[14340]: audit 2024-03-12T02:13:07.767713+0000 mon.smithi153 (mon.0) 313 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-12T02:13:09.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:08 smithi153 bash[14340]: audit 2024-03-12T02:13:07.769109+0000 mon.smithi153 (mon.0) 314 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:09.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:08 smithi153 bash[14340]: cephadm 2024-03-12T02:13:07.770761+0000 mgr.smithi153.asomxr (mgr.14195) 73 : cephadm 1 Reconfiguring daemon mon.smithi163 on smithi163 2024-03-12T02:13:09.122 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:13:09.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: cluster 2024-03-12T02:13:08.320411+0000 mgr.smithi153.asomxr (mgr.14195) 74 : cluster 0 pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:09.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.120395+0000 mon.smithi153 (mon.0) 315 : audit 0 from='client.? 172.21.15.153:0/2097707055' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:13:09.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.512251+0000 mon.smithi153 (mon.0) 316 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.520335+0000 mon.smithi153 (mon.0) 317 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.542701+0000 mon.smithi153 (mon.0) 318 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-12T02:13:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.545284+0000 mon.smithi153 (mon.0) 319 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi153.front.sepia.ceph.com:9093"}]: dispatch 2024-03-12T02:13:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.553675+0000 mon.smithi153 (mon.0) 320 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.567950+0000 mon.smithi153 (mon.0) 321 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-12T02:13:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.571365+0000 mon.smithi153 (mon.0) 322 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi153.front.sepia.ceph.com:3000"}]: dispatch 2024-03-12T02:13:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.580112+0000 mon.smithi153 (mon.0) 323 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.590010+0000 mon.smithi153 (mon.0) 324 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-12T02:13:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.595738+0000 mon.smithi153 (mon.0) 325 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi153.front.sepia.ceph.com:9095"}]: dispatch 2024-03-12T02:13:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.602575+0000 mon.smithi153 (mon.0) 326 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:09 smithi153 bash[14340]: audit 2024-03-12T02:13:09.619184+0000 mon.smithi153 (mon.0) 327 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:13:09.882 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0} 2024-03-12T02:13:09.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: cluster 2024-03-12T02:13:08.320411+0000 mgr.smithi153.asomxr (mgr.14195) 74 : cluster 0 pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:09.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.120395+0000 mon.smithi153 (mon.0) 315 : audit 0 from='client.? 172.21.15.153:0/2097707055' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:13:09.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.512251+0000 mon.smithi153 (mon.0) 316 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.520335+0000 mon.smithi153 (mon.0) 317 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.542701+0000 mon.smithi153 (mon.0) 318 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-12T02:13:09.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.545284+0000 mon.smithi153 (mon.0) 319 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi153.front.sepia.ceph.com:9093"}]: dispatch 2024-03-12T02:13:09.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.553675+0000 mon.smithi153 (mon.0) 320 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.567950+0000 mon.smithi153 (mon.0) 321 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-12T02:13:09.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.571365+0000 mon.smithi153 (mon.0) 322 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi153.front.sepia.ceph.com:3000"}]: dispatch 2024-03-12T02:13:09.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.580112+0000 mon.smithi153 (mon.0) 323 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.590010+0000 mon.smithi153 (mon.0) 324 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-12T02:13:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.595738+0000 mon.smithi153 (mon.0) 325 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi153.front.sepia.ceph.com:9095"}]: dispatch 2024-03-12T02:13:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.602575+0000 mon.smithi153 (mon.0) 326 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:09.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:09 smithi163 bash[22347]: audit 2024-03-12T02:13:09.619184+0000 mon.smithi153 (mon.0) 327 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:13:10.883 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T02:13:11.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:10 smithi153 bash[14340]: audit 2024-03-12T02:13:09.543424+0000 mgr.smithi153.asomxr (mgr.14195) 75 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-12T02:13:11.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:10 smithi153 bash[14340]: audit 2024-03-12T02:13:09.545999+0000 mgr.smithi153.asomxr (mgr.14195) 76 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi153.front.sepia.ceph.com:9093"}]: dispatch 2024-03-12T02:13:11.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:10 smithi153 bash[14340]: audit 2024-03-12T02:13:09.568653+0000 mgr.smithi153.asomxr (mgr.14195) 77 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-12T02:13:11.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:10 smithi153 bash[14340]: audit 2024-03-12T02:13:09.571835+0000 mgr.smithi153.asomxr (mgr.14195) 78 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi153.front.sepia.ceph.com:3000"}]: dispatch 2024-03-12T02:13:11.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:10 smithi153 bash[14340]: audit 2024-03-12T02:13:09.590693+0000 mgr.smithi153.asomxr (mgr.14195) 79 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-12T02:13:11.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:10 smithi153 bash[14340]: audit 2024-03-12T02:13:09.596438+0000 mgr.smithi153.asomxr (mgr.14195) 80 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi153.front.sepia.ceph.com:9095"}]: dispatch 2024-03-12T02:13:11.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:10 smithi163 bash[22347]: audit 2024-03-12T02:13:09.543424+0000 mgr.smithi153.asomxr (mgr.14195) 75 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-12T02:13:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:10 smithi163 bash[22347]: audit 2024-03-12T02:13:09.545999+0000 mgr.smithi153.asomxr (mgr.14195) 76 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi153.front.sepia.ceph.com:9093"}]: dispatch 2024-03-12T02:13:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:10 smithi163 bash[22347]: audit 2024-03-12T02:13:09.568653+0000 mgr.smithi153.asomxr (mgr.14195) 77 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-12T02:13:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:10 smithi163 bash[22347]: audit 2024-03-12T02:13:09.571835+0000 mgr.smithi153.asomxr (mgr.14195) 78 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi153.front.sepia.ceph.com:3000"}]: dispatch 2024-03-12T02:13:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:10 smithi163 bash[22347]: audit 2024-03-12T02:13:09.590693+0000 mgr.smithi153.asomxr (mgr.14195) 79 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-12T02:13:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:10 smithi163 bash[22347]: audit 2024-03-12T02:13:09.596438+0000 mgr.smithi153.asomxr (mgr.14195) 80 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi153.front.sepia.ceph.com:9095"}]: dispatch 2024-03-12T02:13:12.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:11 smithi153 bash[14340]: cluster 2024-03-12T02:13:10.320904+0000 mgr.smithi153.asomxr (mgr.14195) 81 : cluster 0 pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:12.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:11 smithi163 bash[22347]: cluster 2024-03-12T02:13:10.320904+0000 mgr.smithi153.asomxr (mgr.14195) 81 : cluster 0 pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:14.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:13 smithi153 bash[14340]: cluster 2024-03-12T02:13:12.321433+0000 mgr.smithi153.asomxr (mgr.14195) 82 : cluster 0 pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:14.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:13 smithi163 bash[22347]: cluster 2024-03-12T02:13:12.321433+0000 mgr.smithi153.asomxr (mgr.14195) 82 : cluster 0 pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:15.483 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:13:15.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:15 smithi153 bash[14340]: cluster 2024-03-12T02:13:14.321902+0000 mgr.smithi153.asomxr (mgr.14195) 83 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:15.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:15 smithi153 bash[14340]: audit 2024-03-12T02:13:15.058790+0000 mon.smithi153 (mon.0) 328 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:15.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:15 smithi153 bash[14340]: audit 2024-03-12T02:13:15.066130+0000 mon.smithi153 (mon.0) 329 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:15.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:15 smithi153 bash[14340]: audit 2024-03-12T02:13:15.368307+0000 mon.smithi153 (mon.0) 330 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:13:15.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:15 smithi153 bash[14340]: audit 2024-03-12T02:13:15.594819+0000 mon.smithi153 (mon.0) 331 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:15.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:15 smithi153 bash[14340]: audit 2024-03-12T02:13:15.600325+0000 mon.smithi153 (mon.0) 332 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:15.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:15 smithi153 bash[14340]: audit 2024-03-12T02:13:15.710363+0000 mon.smithi153 (mon.0) 333 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:15.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:15 smithi153 bash[14340]: audit 2024-03-12T02:13:15.717659+0000 mon.smithi153 (mon.0) 334 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:16.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:15 smithi163 bash[22347]: cluster 2024-03-12T02:13:14.321902+0000 mgr.smithi153.asomxr (mgr.14195) 83 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:16.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:15 smithi163 bash[22347]: audit 2024-03-12T02:13:15.058790+0000 mon.smithi153 (mon.0) 328 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:16.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:15 smithi163 bash[22347]: audit 2024-03-12T02:13:15.066130+0000 mon.smithi153 (mon.0) 329 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:16.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:15 smithi163 bash[22347]: audit 2024-03-12T02:13:15.368307+0000 mon.smithi153 (mon.0) 330 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:13:16.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:15 smithi163 bash[22347]: audit 2024-03-12T02:13:15.594819+0000 mon.smithi153 (mon.0) 331 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:16.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:15 smithi163 bash[22347]: audit 2024-03-12T02:13:15.600325+0000 mon.smithi153 (mon.0) 332 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:16.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:15 smithi163 bash[22347]: audit 2024-03-12T02:13:15.710363+0000 mon.smithi153 (mon.0) 333 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:16.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:15 smithi163 bash[22347]: audit 2024-03-12T02:13:15.717659+0000 mon.smithi153 (mon.0) 334 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:17.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:17 smithi163 bash[22347]: audit 2024-03-12T02:13:16.201641+0000 mon.smithi153 (mon.0) 335 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:17.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:17 smithi163 bash[22347]: audit 2024-03-12T02:13:16.207705+0000 mon.smithi153 (mon.0) 336 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:17.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:17 smithi163 bash[22347]: audit 2024-03-12T02:13:16.209051+0000 mon.smithi153 (mon.0) 337 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:17.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:17 smithi163 bash[22347]: audit 2024-03-12T02:13:16.209991+0000 mon.smithi153 (mon.0) 338 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:13:17.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:17 smithi163 bash[22347]: audit 2024-03-12T02:13:16.215435+0000 mon.smithi153 (mon.0) 339 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:17.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:17 smithi163 bash[22347]: audit 2024-03-12T02:13:16.219706+0000 mon.smithi153 (mon.0) 340 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:13:17.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:17 smithi163 bash[22347]: audit 2024-03-12T02:13:16.228748+0000 mon.smithi153 (mon.0) 341 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T02:13:17.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:17 smithi163 bash[22347]: audit 2024-03-12T02:13:16.229684+0000 mon.smithi153 (mon.0) 342 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:17.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:17 smithi163 bash[22347]: audit 2024-03-12T02:13:16.233050+0000 mon.smithi153 (mon.0) 343 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T02:13:17.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:17 smithi163 bash[22347]: audit 2024-03-12T02:13:16.233848+0000 mon.smithi153 (mon.0) 344 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:17.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:17 smithi163 bash[22347]: cluster 2024-03-12T02:13:16.322176+0000 mgr.smithi153.asomxr (mgr.14195) 84 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:17.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:17 smithi153 bash[14340]: audit 2024-03-12T02:13:16.201641+0000 mon.smithi153 (mon.0) 335 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:17.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:17 smithi153 bash[14340]: audit 2024-03-12T02:13:16.207705+0000 mon.smithi153 (mon.0) 336 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:17.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:17 smithi153 bash[14340]: audit 2024-03-12T02:13:16.209051+0000 mon.smithi153 (mon.0) 337 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:17.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:17 smithi153 bash[14340]: audit 2024-03-12T02:13:16.209991+0000 mon.smithi153 (mon.0) 338 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:13:17.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:17 smithi153 bash[14340]: audit 2024-03-12T02:13:16.215435+0000 mon.smithi153 (mon.0) 339 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:17.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:17 smithi153 bash[14340]: audit 2024-03-12T02:13:16.219706+0000 mon.smithi153 (mon.0) 340 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:13:17.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:17 smithi153 bash[14340]: audit 2024-03-12T02:13:16.228748+0000 mon.smithi153 (mon.0) 341 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T02:13:17.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:17 smithi153 bash[14340]: audit 2024-03-12T02:13:16.229684+0000 mon.smithi153 (mon.0) 342 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:17.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:17 smithi153 bash[14340]: audit 2024-03-12T02:13:16.233050+0000 mon.smithi153 (mon.0) 343 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T02:13:17.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:17 smithi153 bash[14340]: audit 2024-03-12T02:13:16.233848+0000 mon.smithi153 (mon.0) 344 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:17.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:17 smithi153 bash[14340]: cluster 2024-03-12T02:13:16.322176+0000 mgr.smithi153.asomxr (mgr.14195) 84 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:18.891 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:13:19.700 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0} 2024-03-12T02:13:19.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:19 smithi153 bash[14340]: cluster 2024-03-12T02:13:18.322527+0000 mgr.smithi153.asomxr (mgr.14195) 85 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:19.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:19 smithi153 bash[14340]: audit 2024-03-12T02:13:18.888837+0000 mon.smithi153 (mon.0) 345 : audit 0 from='client.? 172.21.15.153:0/4101933536' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:13:19.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:19 smithi163 bash[22347]: cluster 2024-03-12T02:13:18.322527+0000 mgr.smithi153.asomxr (mgr.14195) 85 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:19.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:19 smithi163 bash[22347]: audit 2024-03-12T02:13:18.888837+0000 mon.smithi153 (mon.0) 345 : audit 0 from='client.? 172.21.15.153:0/4101933536' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:13:20.701 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T02:13:21.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:21 smithi153 bash[14340]: cluster 2024-03-12T02:13:20.322943+0000 mgr.smithi153.asomxr (mgr.14195) 86 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:21.897 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:21 smithi163 bash[22347]: cluster 2024-03-12T02:13:20.322943+0000 mgr.smithi153.asomxr (mgr.14195) 86 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:23.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:23 smithi163 bash[22347]: cluster 2024-03-12T02:13:22.323403+0000 mgr.smithi153.asomxr (mgr.14195) 87 : cluster 0 pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:23.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:23 smithi153 bash[14340]: cluster 2024-03-12T02:13:22.323403+0000 mgr.smithi153.asomxr (mgr.14195) 87 : cluster 0 pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:25.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:25 smithi163 bash[22347]: cluster 2024-03-12T02:13:24.323715+0000 mgr.smithi153.asomxr (mgr.14195) 88 : cluster 0 pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:25.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:25 smithi163 bash[22347]: audit 2024-03-12T02:13:25.207927+0000 mon.smithi163 (mon.1) 2 : audit 1 from='client.? 172.21.15.163:0/189086595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad1d33d5-2e6b-4e4b-b148-3cb45de94e09"}]: dispatch 2024-03-12T02:13:25.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:25 smithi163 bash[22347]: audit 2024-03-12T02:13:25.210803+0000 mon.smithi153 (mon.0) 346 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad1d33d5-2e6b-4e4b-b148-3cb45de94e09"}]: dispatch 2024-03-12T02:13:25.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:25 smithi163 bash[22347]: audit 2024-03-12T02:13:25.219092+0000 mon.smithi153 (mon.0) 347 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad1d33d5-2e6b-4e4b-b148-3cb45de94e09"}]': finished 2024-03-12T02:13:25.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:25 smithi163 bash[22347]: cluster 2024-03-12T02:13:25.219173+0000 mon.smithi153 (mon.0) 348 : cluster 0 osdmap e6: 1 total, 0 up, 1 in 2024-03-12T02:13:25.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:25 smithi163 bash[22347]: audit 2024-03-12T02:13:25.219303+0000 mon.smithi153 (mon.0) 349 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:25.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:25 smithi153 bash[14340]: cluster 2024-03-12T02:13:24.323715+0000 mgr.smithi153.asomxr (mgr.14195) 88 : cluster 0 pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:25.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:25 smithi153 bash[14340]: audit 2024-03-12T02:13:25.207927+0000 mon.smithi163 (mon.1) 2 : audit 1 from='client.? 172.21.15.163:0/189086595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad1d33d5-2e6b-4e4b-b148-3cb45de94e09"}]: dispatch 2024-03-12T02:13:25.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:25 smithi153 bash[14340]: audit 2024-03-12T02:13:25.210803+0000 mon.smithi153 (mon.0) 346 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad1d33d5-2e6b-4e4b-b148-3cb45de94e09"}]: dispatch 2024-03-12T02:13:25.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:25 smithi153 bash[14340]: audit 2024-03-12T02:13:25.219092+0000 mon.smithi153 (mon.0) 347 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad1d33d5-2e6b-4e4b-b148-3cb45de94e09"}]': finished 2024-03-12T02:13:25.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:25 smithi153 bash[14340]: cluster 2024-03-12T02:13:25.219173+0000 mon.smithi153 (mon.0) 348 : cluster 0 osdmap e6: 1 total, 0 up, 1 in 2024-03-12T02:13:25.834 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:25 smithi153 bash[14340]: audit 2024-03-12T02:13:25.219303+0000 mon.smithi153 (mon.0) 349 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:26.125 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:13:26.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:26 smithi163 bash[22347]: audit 2024-03-12T02:13:25.615173+0000 mon.smithi153 (mon.0) 350 : audit 1 from='client.? 172.21.15.153:0/504773337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e789a733-1bcc-4a47-8c1f-6190d10a0dba"}]: dispatch 2024-03-12T02:13:26.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:26 smithi163 bash[22347]: audit 2024-03-12T02:13:25.625467+0000 mon.smithi153 (mon.0) 351 : audit 1 from='client.? 172.21.15.153:0/504773337' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e789a733-1bcc-4a47-8c1f-6190d10a0dba"}]': finished 2024-03-12T02:13:26.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:26 smithi163 bash[22347]: cluster 2024-03-12T02:13:25.625541+0000 mon.smithi153 (mon.0) 352 : cluster 0 osdmap e7: 2 total, 0 up, 2 in 2024-03-12T02:13:26.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:26 smithi163 bash[22347]: audit 2024-03-12T02:13:25.625755+0000 mon.smithi153 (mon.0) 353 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:26.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:26 smithi163 bash[22347]: audit 2024-03-12T02:13:25.626357+0000 mon.smithi153 (mon.0) 354 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:27.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:26 smithi153 bash[14340]: audit 2024-03-12T02:13:25.615173+0000 mon.smithi153 (mon.0) 350 : audit 1 from='client.? 172.21.15.153:0/504773337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e789a733-1bcc-4a47-8c1f-6190d10a0dba"}]: dispatch 2024-03-12T02:13:27.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:26 smithi153 bash[14340]: audit 2024-03-12T02:13:25.625467+0000 mon.smithi153 (mon.0) 351 : audit 1 from='client.? 172.21.15.153:0/504773337' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e789a733-1bcc-4a47-8c1f-6190d10a0dba"}]': finished 2024-03-12T02:13:27.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:26 smithi153 bash[14340]: cluster 2024-03-12T02:13:25.625541+0000 mon.smithi153 (mon.0) 352 : cluster 0 osdmap e7: 2 total, 0 up, 2 in 2024-03-12T02:13:27.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:26 smithi153 bash[14340]: audit 2024-03-12T02:13:25.625755+0000 mon.smithi153 (mon.0) 353 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:27.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:26 smithi153 bash[14340]: audit 2024-03-12T02:13:25.626357+0000 mon.smithi153 (mon.0) 354 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:27.792 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:27 smithi153 bash[14340]: cluster 2024-03-12T02:13:26.324178+0000 mgr.smithi153.asomxr (mgr.14195) 89 : cluster 0 pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:27.792 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:27 smithi153 bash[14340]: audit 2024-03-12T02:13:26.914451+0000 mon.smithi163 (mon.1) 3 : audit 0 from='client.? 172.21.15.163:0/2493798327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:27.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:27 smithi163 bash[22347]: cluster 2024-03-12T02:13:26.324178+0000 mgr.smithi153.asomxr (mgr.14195) 89 : cluster 0 pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:27.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:27 smithi163 bash[22347]: audit 2024-03-12T02:13:26.914451+0000 mon.smithi163 (mon.1) 3 : audit 0 from='client.? 172.21.15.163:0/2493798327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:28.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:28 smithi153 bash[14340]: audit 2024-03-12T02:13:27.973716+0000 mon.smithi153 (mon.0) 355 : audit 0 from='client.? 172.21.15.153:0/1601297309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:28.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:28 smithi163 bash[22347]: audit 2024-03-12T02:13:27.973716+0000 mon.smithi153 (mon.0) 355 : audit 0 from='client.? 172.21.15.153:0/1601297309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:29.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:29 smithi153 bash[14340]: cluster 2024-03-12T02:13:28.324538+0000 mgr.smithi153.asomxr (mgr.14195) 90 : cluster 0 pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:29.909 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:13:29.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:29 smithi163 bash[22347]: cluster 2024-03-12T02:13:28.324538+0000 mgr.smithi153.asomxr (mgr.14195) 90 : cluster 0 pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:30.694 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":8,"num_osds":3,"num_up_osds":0,"osd_up_since":0,"num_in_osds":3,"osd_in_since":1710209609,"num_remapped_pgs":0} 2024-03-12T02:13:30.982 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:30 smithi153 bash[14340]: audit 2024-03-12T02:13:29.795757+0000 mon.smithi163 (mon.1) 4 : audit 1 from='client.? 172.21.15.163:0/3718837744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d17e4690-4fa9-45b1-8fe6-e62f6b2475ef"}]: dispatch 2024-03-12T02:13:30.982 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:30 smithi153 bash[14340]: audit 2024-03-12T02:13:29.798532+0000 mon.smithi153 (mon.0) 356 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d17e4690-4fa9-45b1-8fe6-e62f6b2475ef"}]: dispatch 2024-03-12T02:13:30.983 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:30 smithi153 bash[14340]: audit 2024-03-12T02:13:29.808138+0000 mon.smithi153 (mon.0) 357 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d17e4690-4fa9-45b1-8fe6-e62f6b2475ef"}]': finished 2024-03-12T02:13:30.983 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:30 smithi153 bash[14340]: cluster 2024-03-12T02:13:29.808267+0000 mon.smithi153 (mon.0) 358 : cluster 0 osdmap e8: 3 total, 0 up, 3 in 2024-03-12T02:13:30.983 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:30 smithi153 bash[14340]: audit 2024-03-12T02:13:29.808435+0000 mon.smithi153 (mon.0) 359 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:30.983 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:30 smithi153 bash[14340]: audit 2024-03-12T02:13:29.808938+0000 mon.smithi153 (mon.0) 360 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:30.983 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:30 smithi153 bash[14340]: audit 2024-03-12T02:13:29.809309+0000 mon.smithi153 (mon.0) 361 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:30.983 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:30 smithi153 bash[14340]: audit 2024-03-12T02:13:29.906924+0000 mon.smithi153 (mon.0) 362 : audit 0 from='client.? 172.21.15.153:0/748201840' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:13:30.983 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:30 smithi153 bash[14340]: audit 2024-03-12T02:13:30.368776+0000 mon.smithi153 (mon.0) 363 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:13:30.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:30 smithi163 bash[22347]: audit 2024-03-12T02:13:29.795757+0000 mon.smithi163 (mon.1) 4 : audit 1 from='client.? 172.21.15.163:0/3718837744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d17e4690-4fa9-45b1-8fe6-e62f6b2475ef"}]: dispatch 2024-03-12T02:13:30.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:30 smithi163 bash[22347]: audit 2024-03-12T02:13:29.798532+0000 mon.smithi153 (mon.0) 356 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d17e4690-4fa9-45b1-8fe6-e62f6b2475ef"}]: dispatch 2024-03-12T02:13:30.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:30 smithi163 bash[22347]: audit 2024-03-12T02:13:29.808138+0000 mon.smithi153 (mon.0) 357 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d17e4690-4fa9-45b1-8fe6-e62f6b2475ef"}]': finished 2024-03-12T02:13:30.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:30 smithi163 bash[22347]: cluster 2024-03-12T02:13:29.808267+0000 mon.smithi153 (mon.0) 358 : cluster 0 osdmap e8: 3 total, 0 up, 3 in 2024-03-12T02:13:30.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:30 smithi163 bash[22347]: audit 2024-03-12T02:13:29.808435+0000 mon.smithi153 (mon.0) 359 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:30.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:30 smithi163 bash[22347]: audit 2024-03-12T02:13:29.808938+0000 mon.smithi153 (mon.0) 360 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:30.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:30 smithi163 bash[22347]: audit 2024-03-12T02:13:29.809309+0000 mon.smithi153 (mon.0) 361 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:30.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:30 smithi163 bash[22347]: audit 2024-03-12T02:13:29.906924+0000 mon.smithi153 (mon.0) 362 : audit 0 from='client.? 172.21.15.153:0/748201840' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:13:30.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:30 smithi163 bash[22347]: audit 2024-03-12T02:13:30.368776+0000 mon.smithi153 (mon.0) 363 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:13:31.695 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T02:13:31.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:31 smithi163 bash[22347]: cluster 2024-03-12T02:13:30.324909+0000 mgr.smithi153.asomxr (mgr.14195) 91 : cluster 0 pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:31.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:31 smithi163 bash[22347]: audit 2024-03-12T02:13:30.800796+0000 mon.smithi153 (mon.0) 364 : audit 1 from='client.? 172.21.15.153:0/3555538174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc7e829f-69ae-482f-9633-ba76231f39a9"}]: dispatch 2024-03-12T02:13:31.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:31 smithi163 bash[22347]: audit 2024-03-12T02:13:30.807807+0000 mon.smithi153 (mon.0) 365 : audit 1 from='client.? 172.21.15.153:0/3555538174' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc7e829f-69ae-482f-9633-ba76231f39a9"}]': finished 2024-03-12T02:13:31.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:31 smithi163 bash[22347]: cluster 2024-03-12T02:13:30.807890+0000 mon.smithi153 (mon.0) 366 : cluster 0 osdmap e9: 4 total, 0 up, 4 in 2024-03-12T02:13:31.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:31 smithi163 bash[22347]: audit 2024-03-12T02:13:30.808348+0000 mon.smithi153 (mon.0) 367 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:31.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:31 smithi163 bash[22347]: audit 2024-03-12T02:13:30.808854+0000 mon.smithi153 (mon.0) 368 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:31.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:31 smithi163 bash[22347]: audit 2024-03-12T02:13:30.812383+0000 mon.smithi153 (mon.0) 369 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:31.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:31 smithi163 bash[22347]: audit 2024-03-12T02:13:30.812814+0000 mon.smithi153 (mon.0) 370 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:13:31.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:31 smithi163 bash[22347]: audit 2024-03-12T02:13:31.535969+0000 mon.smithi163 (mon.1) 5 : audit 0 from='client.? 172.21.15.163:0/3744675798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:32.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:31 smithi153 bash[14340]: cluster 2024-03-12T02:13:30.324909+0000 mgr.smithi153.asomxr (mgr.14195) 91 : cluster 0 pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:32.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:31 smithi153 bash[14340]: audit 2024-03-12T02:13:30.800796+0000 mon.smithi153 (mon.0) 364 : audit 1 from='client.? 172.21.15.153:0/3555538174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc7e829f-69ae-482f-9633-ba76231f39a9"}]: dispatch 2024-03-12T02:13:32.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:31 smithi153 bash[14340]: audit 2024-03-12T02:13:30.807807+0000 mon.smithi153 (mon.0) 365 : audit 1 from='client.? 172.21.15.153:0/3555538174' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc7e829f-69ae-482f-9633-ba76231f39a9"}]': finished 2024-03-12T02:13:32.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:31 smithi153 bash[14340]: cluster 2024-03-12T02:13:30.807890+0000 mon.smithi153 (mon.0) 366 : cluster 0 osdmap e9: 4 total, 0 up, 4 in 2024-03-12T02:13:32.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:31 smithi153 bash[14340]: audit 2024-03-12T02:13:30.808348+0000 mon.smithi153 (mon.0) 367 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:32.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:31 smithi153 bash[14340]: audit 2024-03-12T02:13:30.808854+0000 mon.smithi153 (mon.0) 368 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:32.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:31 smithi153 bash[14340]: audit 2024-03-12T02:13:30.812383+0000 mon.smithi153 (mon.0) 369 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:32.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:31 smithi153 bash[14340]: audit 2024-03-12T02:13:30.812814+0000 mon.smithi153 (mon.0) 370 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:13:32.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:31 smithi153 bash[14340]: audit 2024-03-12T02:13:31.535969+0000 mon.smithi163 (mon.1) 5 : audit 0 from='client.? 172.21.15.163:0/3744675798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:32.958 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:32 smithi153 bash[14340]: audit 2024-03-12T02:13:32.437299+0000 mon.smithi153 (mon.0) 371 : audit 0 from='client.? 172.21.15.153:0/2462317530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:32.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:32 smithi163 bash[22347]: audit 2024-03-12T02:13:32.437299+0000 mon.smithi153 (mon.0) 371 : audit 0 from='client.? 172.21.15.153:0/2462317530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:33.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:33 smithi163 bash[22347]: cluster 2024-03-12T02:13:32.325338+0000 mgr.smithi153.asomxr (mgr.14195) 92 : cluster 0 pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:34.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:33 smithi153 bash[14340]: cluster 2024-03-12T02:13:32.325338+0000 mgr.smithi153.asomxr (mgr.14195) 92 : cluster 0 pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:34.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:34 smithi163 bash[22347]: audit 2024-03-12T02:13:34.373455+0000 mon.smithi163 (mon.1) 6 : audit 1 from='client.? 172.21.15.163:0/2050491523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f0bafb-ca9a-47ca-9ec2-1d03004e6efc"}]: dispatch 2024-03-12T02:13:34.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:34 smithi163 bash[22347]: audit 2024-03-12T02:13:34.376192+0000 mon.smithi153 (mon.0) 372 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f0bafb-ca9a-47ca-9ec2-1d03004e6efc"}]: dispatch 2024-03-12T02:13:34.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:34 smithi163 bash[22347]: audit 2024-03-12T02:13:34.384107+0000 mon.smithi153 (mon.0) 373 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40f0bafb-ca9a-47ca-9ec2-1d03004e6efc"}]': finished 2024-03-12T02:13:34.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:34 smithi163 bash[22347]: cluster 2024-03-12T02:13:34.384259+0000 mon.smithi153 (mon.0) 374 : cluster 0 osdmap e10: 5 total, 0 up, 5 in 2024-03-12T02:13:34.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:34 smithi163 bash[22347]: audit 2024-03-12T02:13:34.384483+0000 mon.smithi153 (mon.0) 375 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:34.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:34 smithi163 bash[22347]: audit 2024-03-12T02:13:34.385007+0000 mon.smithi153 (mon.0) 376 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:34.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:34 smithi163 bash[22347]: audit 2024-03-12T02:13:34.385284+0000 mon.smithi153 (mon.0) 377 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:34.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:34 smithi163 bash[22347]: audit 2024-03-12T02:13:34.385623+0000 mon.smithi153 (mon.0) 378 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:13:34.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:34 smithi163 bash[22347]: audit 2024-03-12T02:13:34.385844+0000 mon.smithi153 (mon.0) 379 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:13:35.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:34 smithi153 bash[14340]: audit 2024-03-12T02:13:34.373455+0000 mon.smithi163 (mon.1) 6 : audit 1 from='client.? 172.21.15.163:0/2050491523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f0bafb-ca9a-47ca-9ec2-1d03004e6efc"}]: dispatch 2024-03-12T02:13:35.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:34 smithi153 bash[14340]: audit 2024-03-12T02:13:34.376192+0000 mon.smithi153 (mon.0) 372 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40f0bafb-ca9a-47ca-9ec2-1d03004e6efc"}]: dispatch 2024-03-12T02:13:35.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:34 smithi153 bash[14340]: audit 2024-03-12T02:13:34.384107+0000 mon.smithi153 (mon.0) 373 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40f0bafb-ca9a-47ca-9ec2-1d03004e6efc"}]': finished 2024-03-12T02:13:35.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:34 smithi153 bash[14340]: cluster 2024-03-12T02:13:34.384259+0000 mon.smithi153 (mon.0) 374 : cluster 0 osdmap e10: 5 total, 0 up, 5 in 2024-03-12T02:13:35.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:34 smithi153 bash[14340]: audit 2024-03-12T02:13:34.384483+0000 mon.smithi153 (mon.0) 375 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:35.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:34 smithi153 bash[14340]: audit 2024-03-12T02:13:34.385007+0000 mon.smithi153 (mon.0) 376 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:35.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:34 smithi153 bash[14340]: audit 2024-03-12T02:13:34.385284+0000 mon.smithi153 (mon.0) 377 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:35.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:34 smithi153 bash[14340]: audit 2024-03-12T02:13:34.385623+0000 mon.smithi153 (mon.0) 378 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:13:35.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:34 smithi153 bash[14340]: audit 2024-03-12T02:13:34.385844+0000 mon.smithi153 (mon.0) 379 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:13:35.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:35 smithi153 bash[14340]: cluster 2024-03-12T02:13:34.325849+0000 mgr.smithi153.asomxr (mgr.14195) 93 : cluster 0 pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:35.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:35 smithi153 bash[14340]: audit 2024-03-12T02:13:35.311881+0000 mon.smithi153 (mon.0) 380 : audit 1 from='client.? 172.21.15.153:0/1961403670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "301fd961-da29-4b85-8683-c117bb4a5086"}]: dispatch 2024-03-12T02:13:35.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:35 smithi153 bash[14340]: audit 2024-03-12T02:13:35.321102+0000 mon.smithi153 (mon.0) 381 : audit 1 from='client.? 172.21.15.153:0/1961403670' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "301fd961-da29-4b85-8683-c117bb4a5086"}]': finished 2024-03-12T02:13:35.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:35 smithi153 bash[14340]: cluster 2024-03-12T02:13:35.321214+0000 mon.smithi153 (mon.0) 382 : cluster 0 osdmap e11: 6 total, 0 up, 6 in 2024-03-12T02:13:35.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:35 smithi153 bash[14340]: audit 2024-03-12T02:13:35.321447+0000 mon.smithi153 (mon.0) 383 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:35.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:35 smithi153 bash[14340]: audit 2024-03-12T02:13:35.321934+0000 mon.smithi153 (mon.0) 384 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:35.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:35 smithi153 bash[14340]: audit 2024-03-12T02:13:35.322253+0000 mon.smithi153 (mon.0) 385 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:35.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:35 smithi153 bash[14340]: audit 2024-03-12T02:13:35.322541+0000 mon.smithi153 (mon.0) 386 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:13:35.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:35 smithi153 bash[14340]: audit 2024-03-12T02:13:35.323099+0000 mon.smithi153 (mon.0) 387 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:13:35.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:35 smithi153 bash[14340]: audit 2024-03-12T02:13:35.323408+0000 mon.smithi153 (mon.0) 388 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:13:35.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:35 smithi163 bash[22347]: cluster 2024-03-12T02:13:34.325849+0000 mgr.smithi153.asomxr (mgr.14195) 93 : cluster 0 pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:35.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:35 smithi163 bash[22347]: audit 2024-03-12T02:13:35.311881+0000 mon.smithi153 (mon.0) 380 : audit 1 from='client.? 172.21.15.153:0/1961403670' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "301fd961-da29-4b85-8683-c117bb4a5086"}]: dispatch 2024-03-12T02:13:35.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:35 smithi163 bash[22347]: audit 2024-03-12T02:13:35.321102+0000 mon.smithi153 (mon.0) 381 : audit 1 from='client.? 172.21.15.153:0/1961403670' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "301fd961-da29-4b85-8683-c117bb4a5086"}]': finished 2024-03-12T02:13:35.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:35 smithi163 bash[22347]: cluster 2024-03-12T02:13:35.321214+0000 mon.smithi153 (mon.0) 382 : cluster 0 osdmap e11: 6 total, 0 up, 6 in 2024-03-12T02:13:35.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:35 smithi163 bash[22347]: audit 2024-03-12T02:13:35.321447+0000 mon.smithi153 (mon.0) 383 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:35.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:35 smithi163 bash[22347]: audit 2024-03-12T02:13:35.321934+0000 mon.smithi153 (mon.0) 384 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:35.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:35 smithi163 bash[22347]: audit 2024-03-12T02:13:35.322253+0000 mon.smithi153 (mon.0) 385 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:35.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:35 smithi163 bash[22347]: audit 2024-03-12T02:13:35.322541+0000 mon.smithi153 (mon.0) 386 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:13:35.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:35 smithi163 bash[22347]: audit 2024-03-12T02:13:35.323099+0000 mon.smithi153 (mon.0) 387 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:13:35.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:35 smithi163 bash[22347]: audit 2024-03-12T02:13:35.323408+0000 mon.smithi153 (mon.0) 388 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:13:36.421 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:13:36.820 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:36 smithi153 bash[14340]: audit 2024-03-12T02:13:36.014057+0000 mon.smithi163 (mon.1) 7 : audit 0 from='client.? 172.21.15.163:0/2465707643' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:36.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:36 smithi163 bash[22347]: audit 2024-03-12T02:13:36.014057+0000 mon.smithi163 (mon.1) 7 : audit 0 from='client.? 172.21.15.163:0/2465707643' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:37.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:37 smithi153 bash[14340]: cluster 2024-03-12T02:13:36.326320+0000 mgr.smithi153.asomxr (mgr.14195) 94 : cluster 0 pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:37.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:37 smithi153 bash[14340]: audit 2024-03-12T02:13:36.874530+0000 mon.smithi153 (mon.0) 389 : audit 0 from='client.? 172.21.15.153:0/3476773237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:37.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:37 smithi163 bash[22347]: cluster 2024-03-12T02:13:36.326320+0000 mgr.smithi153.asomxr (mgr.14195) 94 : cluster 0 pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:37.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:37 smithi163 bash[22347]: audit 2024-03-12T02:13:36.874530+0000 mon.smithi153 (mon.0) 389 : audit 0 from='client.? 172.21.15.153:0/3476773237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:39.531 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:13:39.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: cluster 2024-03-12T02:13:38.326645+0000 mgr.smithi153.asomxr (mgr.14195) 95 : cluster 0 pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:39.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: audit 2024-03-12T02:13:38.889396+0000 mon.smithi163 (mon.1) 8 : audit 1 from='client.? 172.21.15.163:0/3235591820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19436c1-f7ac-415b-b4be-1c2c52c87473"}]: dispatch 2024-03-12T02:13:39.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: audit 2024-03-12T02:13:38.892251+0000 mon.smithi153 (mon.0) 390 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19436c1-f7ac-415b-b4be-1c2c52c87473"}]: dispatch 2024-03-12T02:13:39.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: audit 2024-03-12T02:13:38.900304+0000 mon.smithi153 (mon.0) 391 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e19436c1-f7ac-415b-b4be-1c2c52c87473"}]': finished 2024-03-12T02:13:39.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: cluster 2024-03-12T02:13:38.900414+0000 mon.smithi153 (mon.0) 392 : cluster 0 osdmap e12: 7 total, 0 up, 7 in 2024-03-12T02:13:39.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: audit 2024-03-12T02:13:38.900587+0000 mon.smithi153 (mon.0) 393 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:39.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: audit 2024-03-12T02:13:38.901038+0000 mon.smithi153 (mon.0) 394 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: audit 2024-03-12T02:13:38.901395+0000 mon.smithi153 (mon.0) 395 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: audit 2024-03-12T02:13:38.901715+0000 mon.smithi153 (mon.0) 396 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:13:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: audit 2024-03-12T02:13:38.902051+0000 mon.smithi153 (mon.0) 397 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:13:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: audit 2024-03-12T02:13:38.902369+0000 mon.smithi153 (mon.0) 398 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:13:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: audit 2024-03-12T02:13:38.902738+0000 mon.smithi153 (mon.0) 399 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:13:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:39 smithi153 bash[14340]: audit 2024-03-12T02:13:39.529088+0000 mon.smithi153 (mon.0) 400 : audit 0 from='client.? 172.21.15.153:0/543038838' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:13:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: cluster 2024-03-12T02:13:38.326645+0000 mgr.smithi153.asomxr (mgr.14195) 95 : cluster 0 pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: audit 2024-03-12T02:13:38.889396+0000 mon.smithi163 (mon.1) 8 : audit 1 from='client.? 172.21.15.163:0/3235591820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19436c1-f7ac-415b-b4be-1c2c52c87473"}]: dispatch 2024-03-12T02:13:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: audit 2024-03-12T02:13:38.892251+0000 mon.smithi153 (mon.0) 390 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e19436c1-f7ac-415b-b4be-1c2c52c87473"}]: dispatch 2024-03-12T02:13:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: audit 2024-03-12T02:13:38.900304+0000 mon.smithi153 (mon.0) 391 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e19436c1-f7ac-415b-b4be-1c2c52c87473"}]': finished 2024-03-12T02:13:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: cluster 2024-03-12T02:13:38.900414+0000 mon.smithi153 (mon.0) 392 : cluster 0 osdmap e12: 7 total, 0 up, 7 in 2024-03-12T02:13:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: audit 2024-03-12T02:13:38.900587+0000 mon.smithi153 (mon.0) 393 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: audit 2024-03-12T02:13:38.901038+0000 mon.smithi153 (mon.0) 394 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: audit 2024-03-12T02:13:38.901395+0000 mon.smithi153 (mon.0) 395 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: audit 2024-03-12T02:13:38.901715+0000 mon.smithi153 (mon.0) 396 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:13:39.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: audit 2024-03-12T02:13:38.902051+0000 mon.smithi153 (mon.0) 397 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:13:39.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: audit 2024-03-12T02:13:38.902369+0000 mon.smithi153 (mon.0) 398 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:13:39.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: audit 2024-03-12T02:13:38.902738+0000 mon.smithi153 (mon.0) 399 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:13:39.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:39 smithi163 bash[22347]: audit 2024-03-12T02:13:39.529088+0000 mon.smithi153 (mon.0) 400 : audit 0 from='client.? 172.21.15.153:0/543038838' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:13:40.431 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":12,"num_osds":7,"num_up_osds":0,"osd_up_since":0,"num_in_osds":7,"osd_in_since":1710209618,"num_remapped_pgs":0} 2024-03-12T02:13:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: audit 2024-03-12T02:13:39.748051+0000 mon.smithi153 (mon.0) 401 : audit 1 from='client.? 172.21.15.153:0/826092554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a76fe6b3-b2ae-4fab-b922-fd436208b6db"}]: dispatch 2024-03-12T02:13:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: audit 2024-03-12T02:13:39.753807+0000 mon.smithi153 (mon.0) 402 : audit 1 from='client.? 172.21.15.153:0/826092554' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a76fe6b3-b2ae-4fab-b922-fd436208b6db"}]': finished 2024-03-12T02:13:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: cluster 2024-03-12T02:13:39.753848+0000 mon.smithi153 (mon.0) 403 : cluster 0 osdmap e13: 8 total, 0 up, 8 in 2024-03-12T02:13:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: audit 2024-03-12T02:13:39.753984+0000 mon.smithi153 (mon.0) 404 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:40.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: audit 2024-03-12T02:13:39.754473+0000 mon.smithi153 (mon.0) 405 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:40.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: audit 2024-03-12T02:13:39.754733+0000 mon.smithi153 (mon.0) 406 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:40.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: audit 2024-03-12T02:13:39.755016+0000 mon.smithi153 (mon.0) 407 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:13:40.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: audit 2024-03-12T02:13:39.755294+0000 mon.smithi153 (mon.0) 408 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:13:40.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: audit 2024-03-12T02:13:39.755548+0000 mon.smithi153 (mon.0) 409 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:13:40.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: audit 2024-03-12T02:13:39.755803+0000 mon.smithi153 (mon.0) 410 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:13:40.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: audit 2024-03-12T02:13:39.756055+0000 mon.smithi153 (mon.0) 411 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:13:40.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:40 smithi153 bash[14340]: audit 2024-03-12T02:13:40.547541+0000 mon.smithi163 (mon.1) 9 : audit 0 from='client.? 172.21.15.163:0/1159175293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:40.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: audit 2024-03-12T02:13:39.748051+0000 mon.smithi153 (mon.0) 401 : audit 1 from='client.? 172.21.15.153:0/826092554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a76fe6b3-b2ae-4fab-b922-fd436208b6db"}]: dispatch 2024-03-12T02:13:40.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: audit 2024-03-12T02:13:39.753807+0000 mon.smithi153 (mon.0) 402 : audit 1 from='client.? 172.21.15.153:0/826092554' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a76fe6b3-b2ae-4fab-b922-fd436208b6db"}]': finished 2024-03-12T02:13:40.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: cluster 2024-03-12T02:13:39.753848+0000 mon.smithi153 (mon.0) 403 : cluster 0 osdmap e13: 8 total, 0 up, 8 in 2024-03-12T02:13:40.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: audit 2024-03-12T02:13:39.753984+0000 mon.smithi153 (mon.0) 404 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:13:40.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: audit 2024-03-12T02:13:39.754473+0000 mon.smithi153 (mon.0) 405 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:13:40.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: audit 2024-03-12T02:13:39.754733+0000 mon.smithi153 (mon.0) 406 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:13:40.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: audit 2024-03-12T02:13:39.755016+0000 mon.smithi153 (mon.0) 407 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:13:40.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: audit 2024-03-12T02:13:39.755294+0000 mon.smithi153 (mon.0) 408 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:13:40.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: audit 2024-03-12T02:13:39.755548+0000 mon.smithi153 (mon.0) 409 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:13:40.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: audit 2024-03-12T02:13:39.755803+0000 mon.smithi153 (mon.0) 410 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:13:40.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: audit 2024-03-12T02:13:39.756055+0000 mon.smithi153 (mon.0) 411 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:13:40.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:40 smithi163 bash[22347]: audit 2024-03-12T02:13:40.547541+0000 mon.smithi163 (mon.1) 9 : audit 0 from='client.? 172.21.15.163:0/1159175293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:41.432 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T02:13:42.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:41 smithi153 bash[14340]: cluster 2024-03-12T02:13:40.326988+0000 mgr.smithi153.asomxr (mgr.14195) 96 : cluster 0 pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:42.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:41 smithi153 bash[14340]: audit 2024-03-12T02:13:41.727826+0000 mon.smithi153 (mon.0) 412 : audit 0 from='client.? 172.21.15.153:0/3252253438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:42.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:41 smithi163 bash[22347]: cluster 2024-03-12T02:13:40.326988+0000 mgr.smithi153.asomxr (mgr.14195) 96 : cluster 0 pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:42.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:41 smithi163 bash[22347]: audit 2024-03-12T02:13:41.727826+0000 mon.smithi153 (mon.0) 412 : audit 0 from='client.? 172.21.15.153:0/3252253438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-12T02:13:44.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:43 smithi153 bash[14340]: cluster 2024-03-12T02:13:42.327511+0000 mgr.smithi153.asomxr (mgr.14195) 97 : cluster 0 pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:44.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:43 smithi163 bash[22347]: cluster 2024-03-12T02:13:42.327511+0000 mgr.smithi153.asomxr (mgr.14195) 97 : cluster 0 pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:46.022 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:45 smithi153 bash[14340]: cluster 2024-03-12T02:13:44.327935+0000 mgr.smithi153.asomxr (mgr.14195) 98 : cluster 0 pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:46.022 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:45 smithi153 bash[14340]: audit 2024-03-12T02:13:45.369006+0000 mon.smithi153 (mon.0) 413 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:13:46.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:45 smithi163 bash[22347]: cluster 2024-03-12T02:13:44.327935+0000 mgr.smithi153.asomxr (mgr.14195) 98 : cluster 0 pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:46.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:45 smithi163 bash[22347]: audit 2024-03-12T02:13:45.369006+0000 mon.smithi153 (mon.0) 413 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:13:46.571 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:13:48.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:47 smithi153 bash[14340]: cluster 2024-03-12T02:13:46.328310+0000 mgr.smithi153.asomxr (mgr.14195) 99 : cluster 0 pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:48.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:47 smithi163 bash[22347]: cluster 2024-03-12T02:13:46.328310+0000 mgr.smithi153.asomxr (mgr.14195) 99 : cluster 0 pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:49.616 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:13:49.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:49 smithi163 bash[22347]: cluster 2024-03-12T02:13:48.328676+0000 mgr.smithi153.asomxr (mgr.14195) 100 : cluster 0 pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:49.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:49 smithi163 bash[22347]: audit 2024-03-12T02:13:49.614308+0000 mon.smithi153 (mon.0) 414 : audit 0 from='client.? 172.21.15.153:0/2100763564' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:13:50.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:49 smithi153 bash[14340]: cluster 2024-03-12T02:13:48.328676+0000 mgr.smithi153.asomxr (mgr.14195) 100 : cluster 0 pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:50.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:49 smithi153 bash[14340]: audit 2024-03-12T02:13:49.614308+0000 mon.smithi153 (mon.0) 414 : audit 0 from='client.? 172.21.15.153:0/2100763564' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:13:50.894 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":13,"num_osds":8,"num_up_osds":0,"osd_up_since":0,"num_in_osds":8,"osd_in_since":1710209619,"num_remapped_pgs":0} 2024-03-12T02:13:51.895 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T02:13:51.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:51 smithi163 bash[22347]: cluster 2024-03-12T02:13:50.329058+0000 mgr.smithi153.asomxr (mgr.14195) 101 : cluster 0 pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:52.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:51 smithi153 bash[14340]: cluster 2024-03-12T02:13:50.329058+0000 mgr.smithi153.asomxr (mgr.14195) 101 : cluster 0 pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:53.031 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:52 smithi153 bash[14340]: audit 2024-03-12T02:13:52.098729+0000 mon.smithi153 (mon.0) 415 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-12T02:13:53.031 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:52 smithi153 bash[14340]: audit 2024-03-12T02:13:52.099560+0000 mon.smithi153 (mon.0) 416 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:53.199 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:52 smithi163 bash[22347]: audit 2024-03-12T02:13:52.098729+0000 mon.smithi153 (mon.0) 415 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-12T02:13:53.199 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:52 smithi163 bash[22347]: audit 2024-03-12T02:13:52.099560+0000 mon.smithi153 (mon.0) 416 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:54.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:53 smithi153 bash[14340]: cephadm 2024-03-12T02:13:52.100458+0000 mgr.smithi153.asomxr (mgr.14195) 102 : cephadm 1 Deploying daemon osd.0 on smithi163 2024-03-12T02:13:54.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:53 smithi153 bash[14340]: cluster 2024-03-12T02:13:52.329402+0000 mgr.smithi153.asomxr (mgr.14195) 103 : cluster 0 pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:54.093 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:53 smithi163 bash[22347]: cephadm 2024-03-12T02:13:52.100458+0000 mgr.smithi153.asomxr (mgr.14195) 102 : cephadm 1 Deploying daemon osd.0 on smithi163 2024-03-12T02:13:54.093 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:53 smithi163 bash[22347]: cluster 2024-03-12T02:13:52.329402+0000 mgr.smithi153.asomxr (mgr.14195) 103 : cluster 0 pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:55.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:54 smithi153 bash[14340]: audit 2024-03-12T02:13:54.347528+0000 mon.smithi153 (mon.0) 417 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-12T02:13:55.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:54 smithi153 bash[14340]: audit 2024-03-12T02:13:54.348753+0000 mon.smithi153 (mon.0) 418 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:55.181 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:54 smithi163 bash[22347]: audit 2024-03-12T02:13:54.347528+0000 mon.smithi153 (mon.0) 417 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-12T02:13:55.181 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:54 smithi163 bash[22347]: audit 2024-03-12T02:13:54.348753+0000 mon.smithi153 (mon.0) 418 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:55.962 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:55 smithi153 bash[14340]: cluster 2024-03-12T02:13:54.329801+0000 mgr.smithi153.asomxr (mgr.14195) 104 : cluster 0 pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:55.962 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:55 smithi153 bash[14340]: cephadm 2024-03-12T02:13:54.349821+0000 mgr.smithi153.asomxr (mgr.14195) 105 : cephadm 1 Deploying daemon osd.1 on smithi153 2024-03-12T02:13:56.213 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:55 smithi163 bash[22347]: cluster 2024-03-12T02:13:54.329801+0000 mgr.smithi153.asomxr (mgr.14195) 104 : cluster 0 pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:56.213 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:55 smithi163 bash[22347]: cephadm 2024-03-12T02:13:54.349821+0000 mgr.smithi153.asomxr (mgr.14195) 105 : cephadm 1 Deploying daemon osd.1 on smithi153 2024-03-12T02:13:56.972 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:13:57.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:57 smithi163 bash[22347]: cluster 2024-03-12T02:13:56.330213+0000 mgr.smithi153.asomxr (mgr.14195) 106 : cluster 0 pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:57.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:57 smithi163 bash[22347]: audit 2024-03-12T02:13:56.480638+0000 mon.smithi153 (mon.0) 419 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:57.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:57 smithi163 bash[22347]: audit 2024-03-12T02:13:56.487503+0000 mon.smithi153 (mon.0) 420 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:57.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:57 smithi163 bash[22347]: audit 2024-03-12T02:13:56.488828+0000 mon.smithi153 (mon.0) 421 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-12T02:13:57.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:57 smithi163 bash[22347]: audit 2024-03-12T02:13:56.490117+0000 mon.smithi153 (mon.0) 422 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:57.702 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:57 smithi163 bash[22347]: cephadm 2024-03-12T02:13:56.491188+0000 mgr.smithi153.asomxr (mgr.14195) 107 : cephadm 1 Deploying daemon osd.2 on smithi163 2024-03-12T02:13:57.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:57 smithi153 bash[14340]: cluster 2024-03-12T02:13:56.330213+0000 mgr.smithi153.asomxr (mgr.14195) 106 : cluster 0 pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:57.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:57 smithi153 bash[14340]: audit 2024-03-12T02:13:56.480638+0000 mon.smithi153 (mon.0) 419 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:57.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:57 smithi153 bash[14340]: audit 2024-03-12T02:13:56.487503+0000 mon.smithi153 (mon.0) 420 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:13:57.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:57 smithi153 bash[14340]: audit 2024-03-12T02:13:56.488828+0000 mon.smithi153 (mon.0) 421 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-12T02:13:57.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:57 smithi153 bash[14340]: audit 2024-03-12T02:13:56.490117+0000 mon.smithi153 (mon.0) 422 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:13:57.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:57 smithi153 bash[14340]: cephadm 2024-03-12T02:13:56.491188+0000 mgr.smithi153.asomxr (mgr.14195) 107 : cephadm 1 Deploying daemon osd.2 on smithi163 2024-03-12T02:13:59.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:13:59 smithi163 bash[22347]: cluster 2024-03-12T02:13:58.330521+0000 mgr.smithi153.asomxr (mgr.14195) 108 : cluster 0 pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:13:59.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:13:59 smithi153 bash[14340]: cluster 2024-03-12T02:13:58.330521+0000 mgr.smithi153.asomxr (mgr.14195) 108 : cluster 0 pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:14:00.624 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:00 smithi163 bash[22347]: audit 2024-03-12T02:14:00.369326+0000 mon.smithi153 (mon.0) 423 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:14:00.737 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:00 smithi153 bash[14340]: audit 2024-03-12T02:14:00.369326+0000 mon.smithi153 (mon.0) 423 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:14:01.045 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:14:01.803 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:01 smithi163 bash[22347]: cluster 2024-03-12T02:14:00.330795+0000 mgr.smithi153.asomxr (mgr.14195) 109 : cluster 0 pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:14:01.804 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:01 smithi163 bash[22347]: audit 2024-03-12T02:14:00.773665+0000 mon.smithi153 (mon.0) 424 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:01.804 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:01 smithi163 bash[22347]: audit 2024-03-12T02:14:00.781133+0000 mon.smithi153 (mon.0) 425 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:01.804 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:01 smithi163 bash[22347]: audit 2024-03-12T02:14:00.782520+0000 mon.smithi153 (mon.0) 426 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-12T02:14:01.804 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:01 smithi163 bash[22347]: audit 2024-03-12T02:14:00.783429+0000 mon.smithi153 (mon.0) 427 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:01.804 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:01 smithi163 bash[22347]: cephadm 2024-03-12T02:14:00.784206+0000 mgr.smithi153.asomxr (mgr.14195) 110 : cephadm 1 Deploying daemon osd.3 on smithi153 2024-03-12T02:14:01.804 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:01 smithi163 bash[22347]: audit 2024-03-12T02:14:01.043108+0000 mon.smithi153 (mon.0) 428 : audit 0 from='client.? 172.21.15.153:0/3588823119' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:14:02.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:01 smithi153 bash[14340]: cluster 2024-03-12T02:14:00.330795+0000 mgr.smithi153.asomxr (mgr.14195) 109 : cluster 0 pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:14:02.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:01 smithi153 bash[14340]: audit 2024-03-12T02:14:00.773665+0000 mon.smithi153 (mon.0) 424 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:02.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:01 smithi153 bash[14340]: audit 2024-03-12T02:14:00.781133+0000 mon.smithi153 (mon.0) 425 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:02.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:01 smithi153 bash[14340]: audit 2024-03-12T02:14:00.782520+0000 mon.smithi153 (mon.0) 426 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-12T02:14:02.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:01 smithi153 bash[14340]: audit 2024-03-12T02:14:00.783429+0000 mon.smithi153 (mon.0) 427 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:02.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:01 smithi153 bash[14340]: cephadm 2024-03-12T02:14:00.784206+0000 mgr.smithi153.asomxr (mgr.14195) 110 : cephadm 1 Deploying daemon osd.3 on smithi153 2024-03-12T02:14:02.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:01 smithi153 bash[14340]: audit 2024-03-12T02:14:01.043108+0000 mon.smithi153 (mon.0) 428 : audit 0 from='client.? 172.21.15.153:0/3588823119' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:14:02.792 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":13,"num_osds":8,"num_up_osds":0,"osd_up_since":0,"num_in_osds":8,"osd_in_since":1710209619,"num_remapped_pgs":0} 2024-03-12T02:14:03.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:02 smithi153 bash[14340]: audit 2024-03-12T02:14:01.838152+0000 mon.smithi153 (mon.0) 429 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:03.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:02 smithi153 bash[14340]: audit 2024-03-12T02:14:01.844420+0000 mon.smithi153 (mon.0) 430 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:03.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:02 smithi153 bash[14340]: audit 2024-03-12T02:14:01.845607+0000 mon.smithi153 (mon.0) 431 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-12T02:14:03.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:02 smithi153 bash[14340]: audit 2024-03-12T02:14:01.846671+0000 mon.smithi153 (mon.0) 432 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:03.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:02 smithi153 bash[14340]: cephadm 2024-03-12T02:14:01.847609+0000 mgr.smithi153.asomxr (mgr.14195) 111 : cephadm 1 Deploying daemon osd.4 on smithi163 2024-03-12T02:14:03.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:02 smithi163 bash[22347]: audit 2024-03-12T02:14:01.838152+0000 mon.smithi153 (mon.0) 429 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:03.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:02 smithi163 bash[22347]: audit 2024-03-12T02:14:01.844420+0000 mon.smithi153 (mon.0) 430 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:03.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:02 smithi163 bash[22347]: audit 2024-03-12T02:14:01.845607+0000 mon.smithi153 (mon.0) 431 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-12T02:14:03.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:02 smithi163 bash[22347]: audit 2024-03-12T02:14:01.846671+0000 mon.smithi153 (mon.0) 432 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:03.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:02 smithi163 bash[22347]: cephadm 2024-03-12T02:14:01.847609+0000 mgr.smithi153.asomxr (mgr.14195) 111 : cephadm 1 Deploying daemon osd.4 on smithi163 2024-03-12T02:14:03.793 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T02:14:03.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:03 smithi163 bash[22347]: cluster 2024-03-12T02:14:02.331133+0000 mgr.smithi153.asomxr (mgr.14195) 112 : cluster 0 pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:14:04.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:03 smithi153 bash[14340]: cluster 2024-03-12T02:14:02.331133+0000 mgr.smithi153.asomxr (mgr.14195) 112 : cluster 0 pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:14:05.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:04 smithi163 bash[22347]: audit 2024-03-12T02:14:04.026133+0000 mon.smithi163 (mon.1) 10 : audit 1 from='osd.0 [v2:172.21.15.163:6800/3954737342,v1:172.21.15.163:6801/3954737342]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-12T02:14:05.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:04 smithi163 bash[22347]: audit 2024-03-12T02:14:04.028817+0000 mon.smithi153 (mon.0) 433 : audit 1 from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-12T02:14:05.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:04 smithi153 bash[14340]: audit 2024-03-12T02:14:04.026133+0000 mon.smithi163 (mon.1) 10 : audit 1 from='osd.0 [v2:172.21.15.163:6800/3954737342,v1:172.21.15.163:6801/3954737342]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-12T02:14:05.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:04 smithi153 bash[14340]: audit 2024-03-12T02:14:04.028817+0000 mon.smithi153 (mon.0) 433 : audit 1 from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-12T02:14:06.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: cluster 2024-03-12T02:14:04.331593+0000 mgr.smithi153.asomxr (mgr.14195) 113 : cluster 0 pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:14:06.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: audit 2024-03-12T02:14:04.853744+0000 mon.smithi163 (mon.1) 11 : audit 1 from='osd.0 [v2:172.21.15.163:6800/3954737342,v1:172.21.15.163:6801/3954737342]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:06.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: audit 2024-03-12T02:14:04.854617+0000 mon.smithi153 (mon.0) 434 : audit 1 from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-12T02:14:06.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: cluster 2024-03-12T02:14:04.854685+0000 mon.smithi153 (mon.0) 435 : cluster 0 osdmap e14: 8 total, 0 up, 8 in 2024-03-12T02:14:06.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: audit 2024-03-12T02:14:04.854858+0000 mon.smithi153 (mon.0) 436 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:14:06.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: audit 2024-03-12T02:14:04.855172+0000 mon.smithi153 (mon.0) 437 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:06.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: audit 2024-03-12T02:14:04.855345+0000 mon.smithi153 (mon.0) 438 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:06.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: audit 2024-03-12T02:14:04.855508+0000 mon.smithi153 (mon.0) 439 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:06.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: audit 2024-03-12T02:14:04.855664+0000 mon.smithi153 (mon.0) 440 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:06.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: audit 2024-03-12T02:14:04.855961+0000 mon.smithi153 (mon.0) 441 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:06.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: audit 2024-03-12T02:14:04.856223+0000 mon.smithi153 (mon.0) 442 : audit 1 from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:06.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: audit 2024-03-12T02:14:04.856471+0000 mon.smithi153 (mon.0) 443 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:06.195 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:05 smithi163 bash[22347]: audit 2024-03-12T02:14:04.856691+0000 mon.smithi153 (mon.0) 444 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:06.195 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: cluster 2024-03-12T02:14:04.331593+0000 mgr.smithi153.asomxr (mgr.14195) 113 : cluster 0 pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:14:06.195 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: audit 2024-03-12T02:14:04.853744+0000 mon.smithi163 (mon.1) 11 : audit 1 from='osd.0 [v2:172.21.15.163:6800/3954737342,v1:172.21.15.163:6801/3954737342]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:06.195 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: audit 2024-03-12T02:14:04.854617+0000 mon.smithi153 (mon.0) 434 : audit 1 from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-12T02:14:06.195 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: cluster 2024-03-12T02:14:04.854685+0000 mon.smithi153 (mon.0) 435 : cluster 0 osdmap e14: 8 total, 0 up, 8 in 2024-03-12T02:14:06.195 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: audit 2024-03-12T02:14:04.854858+0000 mon.smithi153 (mon.0) 436 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:14:06.195 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: audit 2024-03-12T02:14:04.855172+0000 mon.smithi153 (mon.0) 437 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:06.195 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: audit 2024-03-12T02:14:04.855345+0000 mon.smithi153 (mon.0) 438 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:06.195 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: audit 2024-03-12T02:14:04.855508+0000 mon.smithi153 (mon.0) 439 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:06.195 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: audit 2024-03-12T02:14:04.855664+0000 mon.smithi153 (mon.0) 440 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:06.196 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: audit 2024-03-12T02:14:04.855961+0000 mon.smithi153 (mon.0) 441 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:06.196 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: audit 2024-03-12T02:14:04.856223+0000 mon.smithi153 (mon.0) 442 : audit 1 from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:06.196 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: audit 2024-03-12T02:14:04.856471+0000 mon.smithi153 (mon.0) 443 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:06.196 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:05 smithi153 bash[14340]: audit 2024-03-12T02:14:04.856691+0000 mon.smithi153 (mon.0) 444 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:07.135 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:06 smithi163 bash[22347]: audit 2024-03-12T02:14:05.861042+0000 mon.smithi153 (mon.0) 445 : audit 1 from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-12T02:14:07.135 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:06 smithi163 bash[22347]: cluster 2024-03-12T02:14:05.861250+0000 mon.smithi153 (mon.0) 446 : cluster 0 osdmap e15: 8 total, 0 up, 8 in 2024-03-12T02:14:07.135 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:06 smithi163 bash[22347]: audit 2024-03-12T02:14:05.861687+0000 mon.smithi153 (mon.0) 447 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:14:07.135 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:06 smithi163 bash[22347]: audit 2024-03-12T02:14:05.861961+0000 mon.smithi153 (mon.0) 448 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:07.135 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:06 smithi163 bash[22347]: audit 2024-03-12T02:14:05.862132+0000 mon.smithi153 (mon.0) 449 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:07.136 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:06 smithi163 bash[22347]: audit 2024-03-12T02:14:05.862390+0000 mon.smithi153 (mon.0) 450 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:07.136 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:06 smithi163 bash[22347]: audit 2024-03-12T02:14:05.862560+0000 mon.smithi153 (mon.0) 451 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:07.136 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:06 smithi163 bash[22347]: audit 2024-03-12T02:14:05.862716+0000 mon.smithi153 (mon.0) 452 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:07.136 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:06 smithi163 bash[22347]: audit 2024-03-12T02:14:05.862877+0000 mon.smithi153 (mon.0) 453 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:07.136 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:06 smithi163 bash[22347]: audit 2024-03-12T02:14:05.863042+0000 mon.smithi153 (mon.0) 454 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:07.136 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:06 smithi163 bash[22347]: audit 2024-03-12T02:14:05.870474+0000 mon.smithi153 (mon.0) 455 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:14:07.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:06 smithi153 bash[14340]: audit 2024-03-12T02:14:05.861042+0000 mon.smithi153 (mon.0) 445 : audit 1 from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-12T02:14:07.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:06 smithi153 bash[14340]: cluster 2024-03-12T02:14:05.861250+0000 mon.smithi153 (mon.0) 446 : cluster 0 osdmap e15: 8 total, 0 up, 8 in 2024-03-12T02:14:07.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:06 smithi153 bash[14340]: audit 2024-03-12T02:14:05.861687+0000 mon.smithi153 (mon.0) 447 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:14:07.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:06 smithi153 bash[14340]: audit 2024-03-12T02:14:05.861961+0000 mon.smithi153 (mon.0) 448 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:07.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:06 smithi153 bash[14340]: audit 2024-03-12T02:14:05.862132+0000 mon.smithi153 (mon.0) 449 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:07.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:06 smithi153 bash[14340]: audit 2024-03-12T02:14:05.862390+0000 mon.smithi153 (mon.0) 450 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:07.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:06 smithi153 bash[14340]: audit 2024-03-12T02:14:05.862560+0000 mon.smithi153 (mon.0) 451 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:07.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:06 smithi153 bash[14340]: audit 2024-03-12T02:14:05.862716+0000 mon.smithi153 (mon.0) 452 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:07.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:06 smithi153 bash[14340]: audit 2024-03-12T02:14:05.862877+0000 mon.smithi153 (mon.0) 453 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:07.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:06 smithi153 bash[14340]: audit 2024-03-12T02:14:05.863042+0000 mon.smithi153 (mon.0) 454 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:07.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:06 smithi153 bash[14340]: audit 2024-03-12T02:14:05.870474+0000 mon.smithi153 (mon.0) 455 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:14:08.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: cluster 2024-03-12T02:14:05.021094+0000 osd.0 (osd.0) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: cluster 2024-03-12T02:14:05.021165+0000 osd.0 (osd.0) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: cluster 2024-03-12T02:14:06.331964+0000 mgr.smithi153.asomxr (mgr.14195) 114 : cluster 0 pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:06.871132+0000 mon.smithi153 (mon.0) 456 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: cluster 2024-03-12T02:14:06.879066+0000 mon.smithi153 (mon.0) 457 : cluster 1 osd.0 [v2:172.21.15.163:6800/3954737342,v1:172.21.15.163:6801/3954737342] boot 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: cluster 2024-03-12T02:14:06.879103+0000 mon.smithi153 (mon.0) 458 : cluster 0 osdmap e16: 8 total, 1 up, 8 in 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:06.879302+0000 mon.smithi153 (mon.0) 459 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:06.880372+0000 mon.smithi153 (mon.0) 460 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:06.880801+0000 mon.smithi153 (mon.0) 461 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:06.881265+0000 mon.smithi153 (mon.0) 462 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:06.881696+0000 mon.smithi153 (mon.0) 463 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:06.882111+0000 mon.smithi153 (mon.0) 464 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:06.882640+0000 mon.smithi153 (mon.0) 465 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:06.883021+0000 mon.smithi153 (mon.0) 466 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:08.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:07.658749+0000 mon.smithi153 (mon.0) 467 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:08.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:07.666955+0000 mon.smithi153 (mon.0) 468 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:08.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:07.668237+0000 mon.smithi153 (mon.0) 469 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-12T02:14:08.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:07.669323+0000 mon.smithi153 (mon.0) 470 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:08.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:07.726735+0000 mon.smithi153 (mon.0) 471 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:08.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:07.735724+0000 mon.smithi153 (mon.0) 472 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:08.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:07.736939+0000 mon.smithi153 (mon.0) 473 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-12T02:14:08.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:07 smithi153 bash[14340]: audit 2024-03-12T02:14:07.738086+0000 mon.smithi153 (mon.0) 474 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:08.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: cluster 2024-03-12T02:14:05.021094+0000 osd.0 (osd.0) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:08.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: cluster 2024-03-12T02:14:05.021165+0000 osd.0 (osd.0) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:08.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: cluster 2024-03-12T02:14:06.331964+0000 mgr.smithi153.asomxr (mgr.14195) 114 : cluster 0 pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-12T02:14:08.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:06.871132+0000 mon.smithi153 (mon.0) 456 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:14:08.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: cluster 2024-03-12T02:14:06.879066+0000 mon.smithi153 (mon.0) 457 : cluster 1 osd.0 [v2:172.21.15.163:6800/3954737342,v1:172.21.15.163:6801/3954737342] boot 2024-03-12T02:14:08.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: cluster 2024-03-12T02:14:06.879103+0000 mon.smithi153 (mon.0) 458 : cluster 0 osdmap e16: 8 total, 1 up, 8 in 2024-03-12T02:14:08.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:06.879302+0000 mon.smithi153 (mon.0) 459 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-12T02:14:08.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:06.880372+0000 mon.smithi153 (mon.0) 460 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:08.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:06.880801+0000 mon.smithi153 (mon.0) 461 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:08.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:06.881265+0000 mon.smithi153 (mon.0) 462 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:08.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:06.881696+0000 mon.smithi153 (mon.0) 463 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:08.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:06.882111+0000 mon.smithi153 (mon.0) 464 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:08.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:06.882640+0000 mon.smithi153 (mon.0) 465 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:08.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:06.883021+0000 mon.smithi153 (mon.0) 466 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:08.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:07.658749+0000 mon.smithi153 (mon.0) 467 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:08.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:07.666955+0000 mon.smithi153 (mon.0) 468 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:08.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:07.668237+0000 mon.smithi153 (mon.0) 469 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-12T02:14:08.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:07.669323+0000 mon.smithi153 (mon.0) 470 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:08.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:07.726735+0000 mon.smithi153 (mon.0) 471 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:08.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:07.735724+0000 mon.smithi153 (mon.0) 472 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:08.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:07.736939+0000 mon.smithi153 (mon.0) 473 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-12T02:14:08.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:07 smithi163 bash[22347]: audit 2024-03-12T02:14:07.738086+0000 mon.smithi153 (mon.0) 474 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:08.495 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:14:09.195 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:08 smithi163 bash[22347]: cephadm 2024-03-12T02:14:07.670160+0000 mgr.smithi153.asomxr (mgr.14195) 115 : cephadm 1 Deploying daemon osd.6 on smithi163 2024-03-12T02:14:09.195 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:08 smithi163 bash[22347]: cephadm 2024-03-12T02:14:07.739031+0000 mgr.smithi153.asomxr (mgr.14195) 116 : cephadm 1 Deploying daemon osd.5 on smithi153 2024-03-12T02:14:09.195 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:08 smithi163 bash[22347]: cluster 2024-03-12T02:14:07.886519+0000 mon.smithi153 (mon.0) 475 : cluster 0 osdmap e17: 8 total, 1 up, 8 in 2024-03-12T02:14:09.195 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:08 smithi163 bash[22347]: audit 2024-03-12T02:14:07.886647+0000 mon.smithi153 (mon.0) 476 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:09.195 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:08 smithi163 bash[22347]: audit 2024-03-12T02:14:07.886935+0000 mon.smithi153 (mon.0) 477 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:09.195 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:08 smithi163 bash[22347]: audit 2024-03-12T02:14:07.887131+0000 mon.smithi153 (mon.0) 478 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:09.195 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:08 smithi163 bash[22347]: audit 2024-03-12T02:14:07.887337+0000 mon.smithi153 (mon.0) 479 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:09.195 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:08 smithi163 bash[22347]: audit 2024-03-12T02:14:07.887531+0000 mon.smithi153 (mon.0) 480 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:09.195 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:08 smithi163 bash[22347]: audit 2024-03-12T02:14:07.887727+0000 mon.smithi153 (mon.0) 481 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:09.195 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:08 smithi163 bash[22347]: audit 2024-03-12T02:14:07.887919+0000 mon.smithi153 (mon.0) 482 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:09.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:08 smithi153 bash[14340]: cephadm 2024-03-12T02:14:07.670160+0000 mgr.smithi153.asomxr (mgr.14195) 115 : cephadm 1 Deploying daemon osd.6 on smithi163 2024-03-12T02:14:09.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:08 smithi153 bash[14340]: cephadm 2024-03-12T02:14:07.739031+0000 mgr.smithi153.asomxr (mgr.14195) 116 : cephadm 1 Deploying daemon osd.5 on smithi153 2024-03-12T02:14:09.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:08 smithi153 bash[14340]: cluster 2024-03-12T02:14:07.886519+0000 mon.smithi153 (mon.0) 475 : cluster 0 osdmap e17: 8 total, 1 up, 8 in 2024-03-12T02:14:09.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:08 smithi153 bash[14340]: audit 2024-03-12T02:14:07.886647+0000 mon.smithi153 (mon.0) 476 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:09.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:08 smithi153 bash[14340]: audit 2024-03-12T02:14:07.886935+0000 mon.smithi153 (mon.0) 477 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:09.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:08 smithi153 bash[14340]: audit 2024-03-12T02:14:07.887131+0000 mon.smithi153 (mon.0) 478 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:09.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:08 smithi153 bash[14340]: audit 2024-03-12T02:14:07.887337+0000 mon.smithi153 (mon.0) 479 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:09.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:08 smithi153 bash[14340]: audit 2024-03-12T02:14:07.887531+0000 mon.smithi153 (mon.0) 480 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:09.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:08 smithi153 bash[14340]: audit 2024-03-12T02:14:07.887727+0000 mon.smithi153 (mon.0) 481 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:09.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:08 smithi153 bash[14340]: audit 2024-03-12T02:14:07.887919+0000 mon.smithi153 (mon.0) 482 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:10.127 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:09 smithi153 bash[14340]: cluster 2024-03-12T02:14:08.332472+0000 mgr.smithi153.asomxr (mgr.14195) 117 : cluster 0 pgmap v69: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-03-12T02:14:10.127 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:09 smithi153 bash[14340]: audit 2024-03-12T02:14:09.864722+0000 mon.smithi153 (mon.0) 483 : audit 1 from='osd.1 [v2:172.21.15.153:6802/799977908,v1:172.21.15.153:6803/799977908]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T02:14:10.183 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:09 smithi163 bash[22347]: cluster 2024-03-12T02:14:08.332472+0000 mgr.smithi153.asomxr (mgr.14195) 117 : cluster 0 pgmap v69: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-03-12T02:14:10.184 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:09 smithi163 bash[22347]: audit 2024-03-12T02:14:09.864722+0000 mon.smithi153 (mon.0) 483 : audit 1 from='osd.1 [v2:172.21.15.153:6802/799977908,v1:172.21.15.153:6803/799977908]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-12T02:14:11.018 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: audit 2024-03-12T02:14:09.896463+0000 mon.smithi153 (mon.0) 484 : audit 1 from='osd.1 [v2:172.21.15.153:6802/799977908,v1:172.21.15.153:6803/799977908]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-12T02:14:11.018 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: cluster 2024-03-12T02:14:09.896514+0000 mon.smithi153 (mon.0) 485 : cluster 0 osdmap e18: 8 total, 1 up, 8 in 2024-03-12T02:14:11.018 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: audit 2024-03-12T02:14:09.896711+0000 mon.smithi153 (mon.0) 486 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:11.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: audit 2024-03-12T02:14:09.897404+0000 mon.smithi153 (mon.0) 487 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:11.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: audit 2024-03-12T02:14:09.897879+0000 mon.smithi153 (mon.0) 488 : audit 1 from='osd.1 [v2:172.21.15.153:6802/799977908,v1:172.21.15.153:6803/799977908]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-12T02:14:11.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: audit 2024-03-12T02:14:09.898353+0000 mon.smithi153 (mon.0) 489 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:11.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: audit 2024-03-12T02:14:09.898775+0000 mon.smithi153 (mon.0) 490 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:11.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: audit 2024-03-12T02:14:09.899171+0000 mon.smithi153 (mon.0) 491 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:11.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: audit 2024-03-12T02:14:09.899607+0000 mon.smithi153 (mon.0) 492 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:11.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: audit 2024-03-12T02:14:09.900075+0000 mon.smithi153 (mon.0) 493 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:11.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: audit 2024-03-12T02:14:10.185517+0000 mon.smithi153 (mon.0) 494 : audit 1 from='osd.2 [v2:172.21.15.163:6808/2994034967,v1:172.21.15.163:6809/2994034967]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T02:14:11.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:10 smithi153 bash[14340]: cluster 2024-03-12T02:14:10.332844+0000 mgr.smithi153.asomxr (mgr.14195) 118 : cluster 0 pgmap v71: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-03-12T02:14:11.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: audit 2024-03-12T02:14:09.896463+0000 mon.smithi153 (mon.0) 484 : audit 1 from='osd.1 [v2:172.21.15.153:6802/799977908,v1:172.21.15.153:6803/799977908]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-12T02:14:11.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: cluster 2024-03-12T02:14:09.896514+0000 mon.smithi153 (mon.0) 485 : cluster 0 osdmap e18: 8 total, 1 up, 8 in 2024-03-12T02:14:11.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: audit 2024-03-12T02:14:09.896711+0000 mon.smithi153 (mon.0) 486 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: audit 2024-03-12T02:14:09.897404+0000 mon.smithi153 (mon.0) 487 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: audit 2024-03-12T02:14:09.897879+0000 mon.smithi153 (mon.0) 488 : audit 1 from='osd.1 [v2:172.21.15.153:6802/799977908,v1:172.21.15.153:6803/799977908]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-12T02:14:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: audit 2024-03-12T02:14:09.898353+0000 mon.smithi153 (mon.0) 489 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: audit 2024-03-12T02:14:09.898775+0000 mon.smithi153 (mon.0) 490 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: audit 2024-03-12T02:14:09.899171+0000 mon.smithi153 (mon.0) 491 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: audit 2024-03-12T02:14:09.899607+0000 mon.smithi153 (mon.0) 492 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: audit 2024-03-12T02:14:09.900075+0000 mon.smithi153 (mon.0) 493 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: audit 2024-03-12T02:14:10.185517+0000 mon.smithi153 (mon.0) 494 : audit 1 from='osd.2 [v2:172.21.15.163:6808/2994034967,v1:172.21.15.163:6809/2994034967]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-12T02:14:11.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:10 smithi163 bash[22347]: cluster 2024-03-12T02:14:10.332844+0000 mgr.smithi153.asomxr (mgr.14195) 118 : cluster 0 pgmap v71: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: audit 2024-03-12T02:14:10.903392+0000 mon.smithi153 (mon.0) 495 : audit 1 from='osd.1 [v2:172.21.15.153:6802/799977908,v1:172.21.15.153:6803/799977908]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: audit 2024-03-12T02:14:10.903471+0000 mon.smithi153 (mon.0) 496 : audit 1 from='osd.2 [v2:172.21.15.163:6808/2994034967,v1:172.21.15.163:6809/2994034967]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: cluster 2024-03-12T02:14:10.903528+0000 mon.smithi153 (mon.0) 497 : cluster 0 osdmap e19: 8 total, 1 up, 8 in 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: audit 2024-03-12T02:14:10.904008+0000 mon.smithi153 (mon.0) 498 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: audit 2024-03-12T02:14:10.904686+0000 mon.smithi153 (mon.0) 499 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: audit 2024-03-12T02:14:10.905428+0000 mon.smithi153 (mon.0) 500 : audit 1 from='osd.2 [v2:172.21.15.163:6808/2994034967,v1:172.21.15.163:6809/2994034967]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: audit 2024-03-12T02:14:10.905809+0000 mon.smithi153 (mon.0) 501 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: audit 2024-03-12T02:14:10.906098+0000 mon.smithi153 (mon.0) 502 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: audit 2024-03-12T02:14:10.906367+0000 mon.smithi153 (mon.0) 503 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: audit 2024-03-12T02:14:10.906737+0000 mon.smithi153 (mon.0) 504 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: audit 2024-03-12T02:14:10.907008+0000 mon.smithi153 (mon.0) 505 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:11.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:11 smithi163 bash[22347]: audit 2024-03-12T02:14:10.909680+0000 mon.smithi153 (mon.0) 506 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:12.116 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:11 smithi153 bash[14340]: audit 2024-03-12T02:14:10.903392+0000 mon.smithi153 (mon.0) 495 : audit 1 from='osd.1 [v2:172.21.15.153:6802/799977908,v1:172.21.15.153:6803/799977908]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-12T02:14:12.117 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:11 smithi153 bash[14340]: audit 2024-03-12T02:14:10.903471+0000 mon.smithi153 (mon.0) 496 : audit 1 from='osd.2 [v2:172.21.15.163:6808/2994034967,v1:172.21.15.163:6809/2994034967]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-12T02:14:12.117 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:11 smithi153 bash[14340]: cluster 2024-03-12T02:14:10.903528+0000 mon.smithi153 (mon.0) 497 : cluster 0 osdmap e19: 8 total, 1 up, 8 in 2024-03-12T02:14:12.117 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:11 smithi153 bash[14340]: audit 2024-03-12T02:14:10.904008+0000 mon.smithi153 (mon.0) 498 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:12.117 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:11 smithi153 bash[14340]: audit 2024-03-12T02:14:10.904686+0000 mon.smithi153 (mon.0) 499 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:12.117 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:11 smithi153 bash[14340]: audit 2024-03-12T02:14:10.905428+0000 mon.smithi153 (mon.0) 500 : audit 1 from='osd.2 [v2:172.21.15.163:6808/2994034967,v1:172.21.15.163:6809/2994034967]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:12.117 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:11 smithi153 bash[14340]: audit 2024-03-12T02:14:10.905809+0000 mon.smithi153 (mon.0) 501 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:12.117 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:11 smithi153 bash[14340]: audit 2024-03-12T02:14:10.906098+0000 mon.smithi153 (mon.0) 502 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:12.117 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:11 smithi153 bash[14340]: audit 2024-03-12T02:14:10.906367+0000 mon.smithi153 (mon.0) 503 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:12.117 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:10.906737+0000 mon.smithi153 (mon.0) 504 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:12.117 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:10.907008+0000 mon.smithi153 (mon.0) 505 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:12.117 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:10.909680+0000 mon.smithi153 (mon.0) 506 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:13.191 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: cluster 2024-03-12T02:14:10.860229+0000 osd.1 (osd.1) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:13.191 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: cluster 2024-03-12T02:14:10.860351+0000 osd.1 (osd.1) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:13.191 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: audit 2024-03-12T02:14:11.907642+0000 mon.smithi153 (mon.0) 507 : audit 1 from='osd.2 [v2:172.21.15.163:6808/2994034967,v1:172.21.15.163:6809/2994034967]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-12T02:14:13.191 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: cluster 2024-03-12T02:14:11.907747+0000 mon.smithi153 (mon.0) 508 : cluster 1 osd.1 [v2:172.21.15.153:6802/799977908,v1:172.21.15.153:6803/799977908] boot 2024-03-12T02:14:13.191 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: cluster 2024-03-12T02:14:11.907807+0000 mon.smithi153 (mon.0) 509 : cluster 0 osdmap e20: 8 total, 2 up, 8 in 2024-03-12T02:14:13.191 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: audit 2024-03-12T02:14:11.908220+0000 mon.smithi153 (mon.0) 510 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:13.191 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: audit 2024-03-12T02:14:11.908553+0000 mon.smithi153 (mon.0) 511 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:13.191 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: audit 2024-03-12T02:14:11.909342+0000 mon.smithi153 (mon.0) 512 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:13.192 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: audit 2024-03-12T02:14:11.909555+0000 mon.smithi153 (mon.0) 513 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:13.192 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: audit 2024-03-12T02:14:11.909850+0000 mon.smithi153 (mon.0) 514 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:13.192 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: audit 2024-03-12T02:14:11.910106+0000 mon.smithi153 (mon.0) 515 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:13.192 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: audit 2024-03-12T02:14:11.910327+0000 mon.smithi153 (mon.0) 516 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:13.192 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: audit 2024-03-12T02:14:11.916590+0000 mon.smithi153 (mon.0) 517 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:13.192 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:12 smithi163 bash[22347]: cluster 2024-03-12T02:14:12.333340+0000 mgr.smithi153.asomxr (mgr.14195) 119 : cluster 0 pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: cluster 2024-03-12T02:14:10.860229+0000 osd.1 (osd.1) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: cluster 2024-03-12T02:14:10.860351+0000 osd.1 (osd.1) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:11.907642+0000 mon.smithi153 (mon.0) 507 : audit 1 from='osd.2 [v2:172.21.15.163:6808/2994034967,v1:172.21.15.163:6809/2994034967]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: cluster 2024-03-12T02:14:11.907747+0000 mon.smithi153 (mon.0) 508 : cluster 1 osd.1 [v2:172.21.15.153:6802/799977908,v1:172.21.15.153:6803/799977908] boot 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: cluster 2024-03-12T02:14:11.907807+0000 mon.smithi153 (mon.0) 509 : cluster 0 osdmap e20: 8 total, 2 up, 8 in 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:11.908220+0000 mon.smithi153 (mon.0) 510 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:11.908553+0000 mon.smithi153 (mon.0) 511 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:11.909342+0000 mon.smithi153 (mon.0) 512 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:11.909555+0000 mon.smithi153 (mon.0) 513 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:11.909850+0000 mon.smithi153 (mon.0) 514 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:11.910106+0000 mon.smithi153 (mon.0) 515 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:13.381 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:11.910327+0000 mon.smithi153 (mon.0) 516 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:13.382 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: audit 2024-03-12T02:14:11.916590+0000 mon.smithi153 (mon.0) 517 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:13.382 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:12 smithi153 bash[14340]: cluster 2024-03-12T02:14:12.333340+0000 mgr.smithi153.asomxr (mgr.14195) 119 : cluster 0 pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-12T02:14:14.169 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: cluster 2024-03-12T02:14:11.204647+0000 osd.2 (osd.2) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:14.169 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: cluster 2024-03-12T02:14:11.204893+0000 osd.2 (osd.2) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:14.169 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: audit 2024-03-12T02:14:12.918941+0000 mon.smithi153 (mon.0) 518 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:14.169 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: cluster 2024-03-12T02:14:12.925207+0000 mon.smithi153 (mon.0) 519 : cluster 1 osd.2 [v2:172.21.15.163:6808/2994034967,v1:172.21.15.163:6809/2994034967] boot 2024-03-12T02:14:14.170 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: cluster 2024-03-12T02:14:12.925245+0000 mon.smithi153 (mon.0) 520 : cluster 0 osdmap e21: 8 total, 3 up, 8 in 2024-03-12T02:14:14.170 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: audit 2024-03-12T02:14:12.925356+0000 mon.smithi153 (mon.0) 521 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:14.170 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: audit 2024-03-12T02:14:12.925667+0000 mon.smithi153 (mon.0) 522 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:14.170 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: audit 2024-03-12T02:14:12.925836+0000 mon.smithi153 (mon.0) 523 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:14.170 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: audit 2024-03-12T02:14:12.926046+0000 mon.smithi153 (mon.0) 524 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:14.170 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: audit 2024-03-12T02:14:12.926226+0000 mon.smithi153 (mon.0) 525 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:14.170 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: audit 2024-03-12T02:14:12.926533+0000 mon.smithi153 (mon.0) 526 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:14.170 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: audit 2024-03-12T02:14:13.474498+0000 mon.smithi153 (mon.0) 527 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:14.170 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:13 smithi153 bash[14340]: audit 2024-03-12T02:14:13.491924+0000 mon.smithi153 (mon.0) 528 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:14.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: cluster 2024-03-12T02:14:11.204647+0000 osd.2 (osd.2) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: cluster 2024-03-12T02:14:11.204893+0000 osd.2 (osd.2) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: audit 2024-03-12T02:14:12.918941+0000 mon.smithi153 (mon.0) 518 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: cluster 2024-03-12T02:14:12.925207+0000 mon.smithi153 (mon.0) 519 : cluster 1 osd.2 [v2:172.21.15.163:6808/2994034967,v1:172.21.15.163:6809/2994034967] boot 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: cluster 2024-03-12T02:14:12.925245+0000 mon.smithi153 (mon.0) 520 : cluster 0 osdmap e21: 8 total, 3 up, 8 in 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: audit 2024-03-12T02:14:12.925356+0000 mon.smithi153 (mon.0) 521 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: audit 2024-03-12T02:14:12.925667+0000 mon.smithi153 (mon.0) 522 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: audit 2024-03-12T02:14:12.925836+0000 mon.smithi153 (mon.0) 523 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: audit 2024-03-12T02:14:12.926046+0000 mon.smithi153 (mon.0) 524 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: audit 2024-03-12T02:14:12.926226+0000 mon.smithi153 (mon.0) 525 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: audit 2024-03-12T02:14:12.926533+0000 mon.smithi153 (mon.0) 526 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: audit 2024-03-12T02:14:13.474498+0000 mon.smithi153 (mon.0) 527 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:14.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:13 smithi163 bash[22347]: audit 2024-03-12T02:14:13.491924+0000 mon.smithi153 (mon.0) 528 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:14.895 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:14:15.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:14 smithi153 bash[14340]: cluster 2024-03-12T02:14:13.925936+0000 mon.smithi153 (mon.0) 529 : cluster 0 osdmap e22: 8 total, 3 up, 8 in 2024-03-12T02:14:15.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:14 smithi153 bash[14340]: audit 2024-03-12T02:14:13.926086+0000 mon.smithi153 (mon.0) 530 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:15.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:14 smithi153 bash[14340]: audit 2024-03-12T02:14:13.926447+0000 mon.smithi153 (mon.0) 531 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:15.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:14 smithi153 bash[14340]: audit 2024-03-12T02:14:13.926704+0000 mon.smithi153 (mon.0) 532 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:15.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:14 smithi153 bash[14340]: audit 2024-03-12T02:14:13.926970+0000 mon.smithi153 (mon.0) 533 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:15.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:14 smithi153 bash[14340]: audit 2024-03-12T02:14:13.927235+0000 mon.smithi153 (mon.0) 534 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:15.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:14 smithi153 bash[14340]: cluster 2024-03-12T02:14:14.333711+0000 mgr.smithi153.asomxr (mgr.14195) 120 : cluster 0 pgmap v77: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T02:14:15.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:14 smithi153 bash[14340]: audit 2024-03-12T02:14:14.423342+0000 mon.smithi153 (mon.0) 535 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-12T02:14:15.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:14 smithi153 bash[14340]: audit 2024-03-12T02:14:14.893061+0000 mon.smithi153 (mon.0) 536 : audit 0 from='client.? 172.21.15.153:0/2883386476' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:14:15.202 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:14 smithi163 bash[22347]: cluster 2024-03-12T02:14:13.925936+0000 mon.smithi153 (mon.0) 529 : cluster 0 osdmap e22: 8 total, 3 up, 8 in 2024-03-12T02:14:15.202 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:14 smithi163 bash[22347]: audit 2024-03-12T02:14:13.926086+0000 mon.smithi153 (mon.0) 530 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:15.202 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:14 smithi163 bash[22347]: audit 2024-03-12T02:14:13.926447+0000 mon.smithi153 (mon.0) 531 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:15.202 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:14 smithi163 bash[22347]: audit 2024-03-12T02:14:13.926704+0000 mon.smithi153 (mon.0) 532 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:15.202 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:14 smithi163 bash[22347]: audit 2024-03-12T02:14:13.926970+0000 mon.smithi153 (mon.0) 533 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:15.202 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:14 smithi163 bash[22347]: audit 2024-03-12T02:14:13.927235+0000 mon.smithi153 (mon.0) 534 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:15.202 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:14 smithi163 bash[22347]: cluster 2024-03-12T02:14:14.333711+0000 mgr.smithi153.asomxr (mgr.14195) 120 : cluster 0 pgmap v77: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T02:14:15.202 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:14 smithi163 bash[22347]: audit 2024-03-12T02:14:14.423342+0000 mon.smithi153 (mon.0) 535 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-12T02:14:15.203 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:14 smithi163 bash[22347]: audit 2024-03-12T02:14:14.893061+0000 mon.smithi153 (mon.0) 536 : audit 0 from='client.? 172.21.15.153:0/2883386476' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:14:15.955 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":22,"num_osds":8,"num_up_osds":3,"osd_up_since":1710209652,"num_in_osds":8,"osd_in_since":1710209619,"num_remapped_pgs":0} 2024-03-12T02:14:15.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:15 smithi163 bash[22347]: audit 2024-03-12T02:14:14.935013+0000 mon.smithi153 (mon.0) 537 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-12T02:14:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:15 smithi163 bash[22347]: cluster 2024-03-12T02:14:14.935150+0000 mon.smithi153 (mon.0) 538 : cluster 0 osdmap e23: 8 total, 3 up, 8 in 2024-03-12T02:14:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:15 smithi163 bash[22347]: audit 2024-03-12T02:14:14.935332+0000 mon.smithi153 (mon.0) 539 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:15 smithi163 bash[22347]: audit 2024-03-12T02:14:14.935797+0000 mon.smithi153 (mon.0) 540 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:15 smithi163 bash[22347]: audit 2024-03-12T02:14:14.935971+0000 mon.smithi153 (mon.0) 541 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:15 smithi163 bash[22347]: audit 2024-03-12T02:14:14.936217+0000 mon.smithi153 (mon.0) 542 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:15 smithi163 bash[22347]: audit 2024-03-12T02:14:14.936461+0000 mon.smithi153 (mon.0) 543 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:15 smithi163 bash[22347]: audit 2024-03-12T02:14:14.937312+0000 mon.smithi153 (mon.0) 544 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T02:14:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:15 smithi163 bash[22347]: audit 2024-03-12T02:14:15.370055+0000 mon.smithi153 (mon.0) 545 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:14:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:15 smithi163 bash[22347]: audit 2024-03-12T02:14:15.713304+0000 mon.smithi163 (mon.1) 12 : audit 1 from='osd.4 [v2:172.21.15.163:6816/1978911710,v1:172.21.15.163:6817/1978911710]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T02:14:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:15 smithi163 bash[22347]: audit 2024-03-12T02:14:15.715764+0000 mon.smithi153 (mon.0) 546 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T02:14:16.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:15 smithi153 bash[14340]: audit 2024-03-12T02:14:14.935013+0000 mon.smithi153 (mon.0) 537 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-12T02:14:16.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:15 smithi153 bash[14340]: cluster 2024-03-12T02:14:14.935150+0000 mon.smithi153 (mon.0) 538 : cluster 0 osdmap e23: 8 total, 3 up, 8 in 2024-03-12T02:14:16.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:15 smithi153 bash[14340]: audit 2024-03-12T02:14:14.935332+0000 mon.smithi153 (mon.0) 539 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:16.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:15 smithi153 bash[14340]: audit 2024-03-12T02:14:14.935797+0000 mon.smithi153 (mon.0) 540 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:16.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:15 smithi153 bash[14340]: audit 2024-03-12T02:14:14.935971+0000 mon.smithi153 (mon.0) 541 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:15 smithi153 bash[14340]: audit 2024-03-12T02:14:14.936217+0000 mon.smithi153 (mon.0) 542 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:15 smithi153 bash[14340]: audit 2024-03-12T02:14:14.936461+0000 mon.smithi153 (mon.0) 543 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:15 smithi153 bash[14340]: audit 2024-03-12T02:14:14.937312+0000 mon.smithi153 (mon.0) 544 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-12T02:14:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:15 smithi153 bash[14340]: audit 2024-03-12T02:14:15.370055+0000 mon.smithi153 (mon.0) 545 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:14:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:15 smithi153 bash[14340]: audit 2024-03-12T02:14:15.713304+0000 mon.smithi163 (mon.1) 12 : audit 1 from='osd.4 [v2:172.21.15.163:6816/1978911710,v1:172.21.15.163:6817/1978911710]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T02:14:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:15 smithi153 bash[14340]: audit 2024-03-12T02:14:15.715764+0000 mon.smithi153 (mon.0) 546 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-12T02:14:16.957 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T02:14:17.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:15.942104+0000 mon.smithi163 (mon.1) 13 : audit 1 from='osd.4 [v2:172.21.15.163:6816/1978911710,v1:172.21.15.163:6817/1978911710]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:17.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:15.942968+0000 mon.smithi153 (mon.0) 547 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-12T02:14:17.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:15.943032+0000 mon.smithi153 (mon.0) 548 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-12T02:14:17.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: cluster 2024-03-12T02:14:15.943086+0000 mon.smithi153 (mon.0) 549 : cluster 0 osdmap e24: 8 total, 3 up, 8 in 2024-03-12T02:14:17.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:15.943294+0000 mon.smithi153 (mon.0) 550 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:17.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:15.943749+0000 mon.smithi153 (mon.0) 551 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:17.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:15.943996+0000 mon.smithi153 (mon.0) 552 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:17.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:15.944407+0000 mon.smithi153 (mon.0) 553 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:17.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:15.944728+0000 mon.smithi153 (mon.0) 554 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:17.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:15.945134+0000 mon.smithi153 (mon.0) 555 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:17.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: cluster 2024-03-12T02:14:16.334183+0000 mgr.smithi153.asomxr (mgr.14195) 121 : cluster 0 pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T02:14:17.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:16.639019+0000 mon.smithi153 (mon.0) 556 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:17.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:16.646676+0000 mon.smithi153 (mon.0) 557 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:17.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:16.648260+0000 mon.smithi153 (mon.0) 558 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-12T02:14:17.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: audit 2024-03-12T02:14:16.649357+0000 mon.smithi153 (mon.0) 559 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:17.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:16 smithi163 bash[22347]: cephadm 2024-03-12T02:14:16.650283+0000 mgr.smithi153.asomxr (mgr.14195) 122 : cephadm 1 Deploying daemon osd.7 on smithi153 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:15.942104+0000 mon.smithi163 (mon.1) 13 : audit 1 from='osd.4 [v2:172.21.15.163:6816/1978911710,v1:172.21.15.163:6817/1978911710]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:15.942968+0000 mon.smithi153 (mon.0) 547 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:15.943032+0000 mon.smithi153 (mon.0) 548 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: cluster 2024-03-12T02:14:15.943086+0000 mon.smithi153 (mon.0) 549 : cluster 0 osdmap e24: 8 total, 3 up, 8 in 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:15.943294+0000 mon.smithi153 (mon.0) 550 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:15.943749+0000 mon.smithi153 (mon.0) 551 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:15.943996+0000 mon.smithi153 (mon.0) 552 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:15.944407+0000 mon.smithi153 (mon.0) 553 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:15.944728+0000 mon.smithi153 (mon.0) 554 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:15.945134+0000 mon.smithi153 (mon.0) 555 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: cluster 2024-03-12T02:14:16.334183+0000 mgr.smithi153.asomxr (mgr.14195) 121 : cluster 0 pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-12T02:14:17.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:16.639019+0000 mon.smithi153 (mon.0) 556 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:17.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:16.646676+0000 mon.smithi153 (mon.0) 557 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:17.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:16.648260+0000 mon.smithi153 (mon.0) 558 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-12T02:14:17.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: audit 2024-03-12T02:14:16.649357+0000 mon.smithi153 (mon.0) 559 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:17.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:16 smithi153 bash[14340]: cephadm 2024-03-12T02:14:16.650283+0000 mgr.smithi153.asomxr (mgr.14195) 122 : cephadm 1 Deploying daemon osd.7 on smithi153 2024-03-12T02:14:18.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:17 smithi163 bash[22347]: audit 2024-03-12T02:14:16.950583+0000 mon.smithi153 (mon.0) 560 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-12T02:14:18.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:17 smithi163 bash[22347]: cluster 2024-03-12T02:14:16.950709+0000 mon.smithi153 (mon.0) 561 : cluster 0 osdmap e25: 8 total, 3 up, 8 in 2024-03-12T02:14:18.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:17 smithi163 bash[22347]: audit 2024-03-12T02:14:16.951167+0000 mon.smithi153 (mon.0) 562 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:18.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:17 smithi163 bash[22347]: audit 2024-03-12T02:14:16.951767+0000 mon.smithi153 (mon.0) 563 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:18.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:17 smithi163 bash[22347]: audit 2024-03-12T02:14:16.952034+0000 mon.smithi153 (mon.0) 564 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:18.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:17 smithi163 bash[22347]: audit 2024-03-12T02:14:16.952238+0000 mon.smithi153 (mon.0) 565 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:18.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:17 smithi163 bash[22347]: audit 2024-03-12T02:14:16.952419+0000 mon.smithi153 (mon.0) 566 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:18.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:17 smithi163 bash[22347]: audit 2024-03-12T02:14:16.958237+0000 mon.smithi153 (mon.0) 567 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:18.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:17 smithi153 bash[14340]: audit 2024-03-12T02:14:16.950583+0000 mon.smithi153 (mon.0) 560 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-12T02:14:18.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:17 smithi153 bash[14340]: cluster 2024-03-12T02:14:16.950709+0000 mon.smithi153 (mon.0) 561 : cluster 0 osdmap e25: 8 total, 3 up, 8 in 2024-03-12T02:14:18.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:17 smithi153 bash[14340]: audit 2024-03-12T02:14:16.951167+0000 mon.smithi153 (mon.0) 562 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:18.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:17 smithi153 bash[14340]: audit 2024-03-12T02:14:16.951767+0000 mon.smithi153 (mon.0) 563 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:18.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:17 smithi153 bash[14340]: audit 2024-03-12T02:14:16.952034+0000 mon.smithi153 (mon.0) 564 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:18.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:17 smithi153 bash[14340]: audit 2024-03-12T02:14:16.952238+0000 mon.smithi153 (mon.0) 565 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:18.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:17 smithi153 bash[14340]: audit 2024-03-12T02:14:16.952419+0000 mon.smithi153 (mon.0) 566 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:18.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:17 smithi153 bash[14340]: audit 2024-03-12T02:14:16.958237+0000 mon.smithi153 (mon.0) 567 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:19.145 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: cluster 2024-03-12T02:14:16.681626+0000 osd.4 (osd.4) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:19.145 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: cluster 2024-03-12T02:14:16.681759+0000 osd.4 (osd.4) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:19.145 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: cluster 2024-03-12T02:14:17.951512+0000 mon.smithi153 (mon.0) 568 : cluster 1 osd.4 [v2:172.21.15.163:6816/1978911710,v1:172.21.15.163:6817/1978911710] boot 2024-03-12T02:14:19.146 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: cluster 2024-03-12T02:14:17.951571+0000 mon.smithi153 (mon.0) 569 : cluster 0 osdmap e26: 8 total, 4 up, 8 in 2024-03-12T02:14:19.146 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: audit 2024-03-12T02:14:17.952277+0000 mon.smithi153 (mon.0) 570 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:19.146 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: audit 2024-03-12T02:14:17.952527+0000 mon.smithi153 (mon.0) 571 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:19.146 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: audit 2024-03-12T02:14:17.952913+0000 mon.smithi153 (mon.0) 572 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:19.146 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: audit 2024-03-12T02:14:17.953158+0000 mon.smithi153 (mon.0) 573 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:19.146 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: audit 2024-03-12T02:14:17.953433+0000 mon.smithi153 (mon.0) 574 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:19.146 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: cluster 2024-03-12T02:14:17.970693+0000 mon.smithi153 (mon.0) 575 : cluster 0 mgrmap e19: smithi153.asomxr(active, since 2m), standbys: smithi163.fvmiae 2024-03-12T02:14:19.146 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: audit 2024-03-12T02:14:17.986057+0000 mon.smithi153 (mon.0) 576 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:19.146 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: audit 2024-03-12T02:14:18.109761+0000 mon.smithi153 (mon.0) 577 : audit 1 from='osd.3 [v2:172.21.15.153:6810/347906783,v1:172.21.15.153:6811/347906783]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-12T02:14:19.146 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: cluster 2024-03-12T02:14:18.334649+0000 mgr.smithi153.asomxr (mgr.14195) 123 : cluster 0 pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-03-12T02:14:19.146 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:18 smithi153 bash[14340]: audit 2024-03-12T02:14:18.949261+0000 mon.smithi153 (mon.0) 578 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:19.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: cluster 2024-03-12T02:14:16.681626+0000 osd.4 (osd.4) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: cluster 2024-03-12T02:14:16.681759+0000 osd.4 (osd.4) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: cluster 2024-03-12T02:14:17.951512+0000 mon.smithi153 (mon.0) 568 : cluster 1 osd.4 [v2:172.21.15.163:6816/1978911710,v1:172.21.15.163:6817/1978911710] boot 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: cluster 2024-03-12T02:14:17.951571+0000 mon.smithi153 (mon.0) 569 : cluster 0 osdmap e26: 8 total, 4 up, 8 in 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: audit 2024-03-12T02:14:17.952277+0000 mon.smithi153 (mon.0) 570 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: audit 2024-03-12T02:14:17.952527+0000 mon.smithi153 (mon.0) 571 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: audit 2024-03-12T02:14:17.952913+0000 mon.smithi153 (mon.0) 572 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: audit 2024-03-12T02:14:17.953158+0000 mon.smithi153 (mon.0) 573 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: audit 2024-03-12T02:14:17.953433+0000 mon.smithi153 (mon.0) 574 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: cluster 2024-03-12T02:14:17.970693+0000 mon.smithi153 (mon.0) 575 : cluster 0 mgrmap e19: smithi153.asomxr(active, since 2m), standbys: smithi163.fvmiae 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: audit 2024-03-12T02:14:17.986057+0000 mon.smithi153 (mon.0) 576 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: audit 2024-03-12T02:14:18.109761+0000 mon.smithi153 (mon.0) 577 : audit 1 from='osd.3 [v2:172.21.15.153:6810/347906783,v1:172.21.15.153:6811/347906783]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: cluster 2024-03-12T02:14:18.334649+0000 mgr.smithi153.asomxr (mgr.14195) 123 : cluster 0 pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-03-12T02:14:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:18 smithi163 bash[22347]: audit 2024-03-12T02:14:18.949261+0000 mon.smithi153 (mon.0) 578 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:20.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:19 smithi153 bash[14340]: audit 2024-03-12T02:14:18.992866+0000 mon.smithi153 (mon.0) 579 : audit 1 from='osd.3 [v2:172.21.15.153:6810/347906783,v1:172.21.15.153:6811/347906783]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-12T02:14:20.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:19 smithi153 bash[14340]: cluster 2024-03-12T02:14:18.992943+0000 mon.smithi153 (mon.0) 580 : cluster 0 osdmap e27: 8 total, 4 up, 8 in 2024-03-12T02:14:20.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:19 smithi153 bash[14340]: audit 2024-03-12T02:14:18.993067+0000 mon.smithi153 (mon.0) 581 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:20.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:19 smithi153 bash[14340]: audit 2024-03-12T02:14:18.993398+0000 mon.smithi153 (mon.0) 582 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:20.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:19 smithi153 bash[14340]: audit 2024-03-12T02:14:18.993557+0000 mon.smithi153 (mon.0) 583 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:20.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:19 smithi153 bash[14340]: audit 2024-03-12T02:14:18.993761+0000 mon.smithi153 (mon.0) 584 : audit 1 from='osd.3 [v2:172.21.15.153:6810/347906783,v1:172.21.15.153:6811/347906783]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-12T02:14:20.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:19 smithi153 bash[14340]: audit 2024-03-12T02:14:18.993982+0000 mon.smithi153 (mon.0) 585 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:20.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:20 smithi153 bash[14340]: audit 2024-03-12T02:14:19.903334+0000 mon.smithi153 (mon.0) 586 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:20.415 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:19 smithi163 bash[22347]: audit 2024-03-12T02:14:18.992866+0000 mon.smithi153 (mon.0) 579 : audit 1 from='osd.3 [v2:172.21.15.153:6810/347906783,v1:172.21.15.153:6811/347906783]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-12T02:14:20.416 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:19 smithi163 bash[22347]: cluster 2024-03-12T02:14:18.992943+0000 mon.smithi153 (mon.0) 580 : cluster 0 osdmap e27: 8 total, 4 up, 8 in 2024-03-12T02:14:20.416 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:19 smithi163 bash[22347]: audit 2024-03-12T02:14:18.993067+0000 mon.smithi153 (mon.0) 581 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:20.416 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:19 smithi163 bash[22347]: audit 2024-03-12T02:14:18.993398+0000 mon.smithi153 (mon.0) 582 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:20.416 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:19 smithi163 bash[22347]: audit 2024-03-12T02:14:18.993557+0000 mon.smithi153 (mon.0) 583 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:20.416 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:19 smithi163 bash[22347]: audit 2024-03-12T02:14:18.993761+0000 mon.smithi153 (mon.0) 584 : audit 1 from='osd.3 [v2:172.21.15.153:6810/347906783,v1:172.21.15.153:6811/347906783]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-12T02:14:20.416 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:19 smithi163 bash[22347]: audit 2024-03-12T02:14:18.993982+0000 mon.smithi153 (mon.0) 585 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:20.416 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:19 smithi163 bash[22347]: audit 2024-03-12T02:14:19.903334+0000 mon.smithi153 (mon.0) 586 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:21.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:21 smithi163 bash[22347]: audit 2024-03-12T02:14:19.997452+0000 mon.smithi153 (mon.0) 587 : audit 1 from='osd.3 [v2:172.21.15.153:6810/347906783,v1:172.21.15.153:6811/347906783]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-12T02:14:21.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:21 smithi163 bash[22347]: cluster 2024-03-12T02:14:19.997635+0000 mon.smithi153 (mon.0) 588 : cluster 0 osdmap e28: 8 total, 4 up, 8 in 2024-03-12T02:14:21.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:21 smithi163 bash[22347]: audit 2024-03-12T02:14:19.998311+0000 mon.smithi153 (mon.0) 589 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:21.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:21 smithi163 bash[22347]: audit 2024-03-12T02:14:19.999123+0000 mon.smithi153 (mon.0) 590 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:21.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:21 smithi163 bash[22347]: audit 2024-03-12T02:14:20.004037+0000 mon.smithi153 (mon.0) 591 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:21.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:21 smithi163 bash[22347]: audit 2024-03-12T02:14:20.004624+0000 mon.smithi153 (mon.0) 592 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:21.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:21 smithi163 bash[22347]: audit 2024-03-12T02:14:20.009370+0000 mon.smithi153 (mon.0) 593 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:21.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:21 smithi163 bash[22347]: cluster 2024-03-12T02:14:20.335032+0000 mgr.smithi153.asomxr (mgr.14195) 124 : cluster 0 pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 357 GiB / 358 GiB avail; 906 B/s rd, 154 KiB/s wr, 8 op/s 2024-03-12T02:14:21.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:21 smithi163 bash[22347]: audit 2024-03-12T02:14:20.416927+0000 mon.smithi163 (mon.1) 14 : audit 1 from='osd.6 [v2:172.21.15.163:6824/1178116342,v1:172.21.15.163:6825/1178116342]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-12T02:14:21.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:21 smithi163 bash[22347]: audit 2024-03-12T02:14:20.419114+0000 mon.smithi153 (mon.0) 594 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-12T02:14:21.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:21 smithi153 bash[14340]: audit 2024-03-12T02:14:19.997452+0000 mon.smithi153 (mon.0) 587 : audit 1 from='osd.3 [v2:172.21.15.153:6810/347906783,v1:172.21.15.153:6811/347906783]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-12T02:14:21.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:21 smithi153 bash[14340]: cluster 2024-03-12T02:14:19.997635+0000 mon.smithi153 (mon.0) 588 : cluster 0 osdmap e28: 8 total, 4 up, 8 in 2024-03-12T02:14:21.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:21 smithi153 bash[14340]: audit 2024-03-12T02:14:19.998311+0000 mon.smithi153 (mon.0) 589 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:21.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:21 smithi153 bash[14340]: audit 2024-03-12T02:14:19.999123+0000 mon.smithi153 (mon.0) 590 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:21.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:21 smithi153 bash[14340]: audit 2024-03-12T02:14:20.004037+0000 mon.smithi153 (mon.0) 591 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:21.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:21 smithi153 bash[14340]: audit 2024-03-12T02:14:20.004624+0000 mon.smithi153 (mon.0) 592 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:21.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:21 smithi153 bash[14340]: audit 2024-03-12T02:14:20.009370+0000 mon.smithi153 (mon.0) 593 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:21.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:21 smithi153 bash[14340]: cluster 2024-03-12T02:14:20.335032+0000 mgr.smithi153.asomxr (mgr.14195) 124 : cluster 0 pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 357 GiB / 358 GiB avail; 906 B/s rd, 154 KiB/s wr, 8 op/s 2024-03-12T02:14:21.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:21 smithi153 bash[14340]: audit 2024-03-12T02:14:20.416927+0000 mon.smithi163 (mon.1) 14 : audit 1 from='osd.6 [v2:172.21.15.163:6824/1178116342,v1:172.21.15.163:6825/1178116342]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-12T02:14:21.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:21 smithi153 bash[14340]: audit 2024-03-12T02:14:20.419114+0000 mon.smithi153 (mon.0) 594 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-12T02:14:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:22 smithi163 bash[22347]: cluster 2024-03-12T02:14:19.155061+0000 osd.3 (osd.3) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:22 smithi163 bash[22347]: cluster 2024-03-12T02:14:19.155147+0000 osd.3 (osd.3) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:22 smithi163 bash[22347]: audit 2024-03-12T02:14:21.009819+0000 mon.smithi163 (mon.1) 15 : audit 1 from='osd.6 [v2:172.21.15.163:6824/1178116342,v1:172.21.15.163:6825/1178116342]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:22 smithi163 bash[22347]: cluster 2024-03-12T02:14:21.010487+0000 mon.smithi153 (mon.0) 595 : cluster 1 osd.3 [v2:172.21.15.153:6810/347906783,v1:172.21.15.153:6811/347906783] boot 2024-03-12T02:14:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:22 smithi163 bash[22347]: audit 2024-03-12T02:14:21.010579+0000 mon.smithi153 (mon.0) 596 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-12T02:14:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:22 smithi163 bash[22347]: cluster 2024-03-12T02:14:21.010669+0000 mon.smithi153 (mon.0) 597 : cluster 0 osdmap e29: 8 total, 5 up, 8 in 2024-03-12T02:14:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:22 smithi163 bash[22347]: audit 2024-03-12T02:14:21.011386+0000 mon.smithi153 (mon.0) 598 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:22 smithi163 bash[22347]: audit 2024-03-12T02:14:21.011889+0000 mon.smithi153 (mon.0) 599 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:22 smithi163 bash[22347]: audit 2024-03-12T02:14:21.012429+0000 mon.smithi153 (mon.0) 600 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:22 smithi163 bash[22347]: audit 2024-03-12T02:14:21.012795+0000 mon.smithi153 (mon.0) 601 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:22.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:22 smithi163 bash[22347]: audit 2024-03-12T02:14:21.013401+0000 mon.smithi153 (mon.0) 602 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:22.248 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:22 smithi153 bash[14340]: cluster 2024-03-12T02:14:19.155061+0000 osd.3 (osd.3) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:22.249 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:22 smithi153 bash[14340]: cluster 2024-03-12T02:14:19.155147+0000 osd.3 (osd.3) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:22.249 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:22 smithi153 bash[14340]: audit 2024-03-12T02:14:21.009819+0000 mon.smithi163 (mon.1) 15 : audit 1 from='osd.6 [v2:172.21.15.163:6824/1178116342,v1:172.21.15.163:6825/1178116342]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:22.249 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:22 smithi153 bash[14340]: cluster 2024-03-12T02:14:21.010487+0000 mon.smithi153 (mon.0) 595 : cluster 1 osd.3 [v2:172.21.15.153:6810/347906783,v1:172.21.15.153:6811/347906783] boot 2024-03-12T02:14:22.249 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:22 smithi153 bash[14340]: audit 2024-03-12T02:14:21.010579+0000 mon.smithi153 (mon.0) 596 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-12T02:14:22.249 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:22 smithi153 bash[14340]: cluster 2024-03-12T02:14:21.010669+0000 mon.smithi153 (mon.0) 597 : cluster 0 osdmap e29: 8 total, 5 up, 8 in 2024-03-12T02:14:22.249 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:22 smithi153 bash[14340]: audit 2024-03-12T02:14:21.011386+0000 mon.smithi153 (mon.0) 598 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-12T02:14:22.249 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:22 smithi153 bash[14340]: audit 2024-03-12T02:14:21.011889+0000 mon.smithi153 (mon.0) 599 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:22.249 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:22 smithi153 bash[14340]: audit 2024-03-12T02:14:21.012429+0000 mon.smithi153 (mon.0) 600 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:22.249 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:22 smithi153 bash[14340]: audit 2024-03-12T02:14:21.012795+0000 mon.smithi153 (mon.0) 601 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]: dispatch 2024-03-12T02:14:22.249 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:22 smithi153 bash[14340]: audit 2024-03-12T02:14:21.013401+0000 mon.smithi153 (mon.0) 602 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:23.292 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:23 smithi153 bash[14340]: audit 2024-03-12T02:14:22.014546+0000 mon.smithi153 (mon.0) 603 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-12T02:14:23.292 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:23 smithi153 bash[14340]: cluster 2024-03-12T02:14:22.014659+0000 mon.smithi153 (mon.0) 604 : cluster 0 osdmap e30: 8 total, 5 up, 8 in 2024-03-12T02:14:23.292 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:23 smithi153 bash[14340]: audit 2024-03-12T02:14:22.014796+0000 mon.smithi153 (mon.0) 605 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:23.292 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:23 smithi153 bash[14340]: audit 2024-03-12T02:14:22.015313+0000 mon.smithi153 (mon.0) 606 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:23.292 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:23 smithi153 bash[14340]: audit 2024-03-12T02:14:22.015601+0000 mon.smithi153 (mon.0) 607 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:23.292 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:23 smithi153 bash[14340]: audit 2024-03-12T02:14:22.019805+0000 mon.smithi153 (mon.0) 608 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:23.292 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:23 smithi153 bash[14340]: cluster 2024-03-12T02:14:22.335643+0000 mgr.smithi153.asomxr (mgr.14195) 125 : cluster 0 pgmap v89: 1 pgs: 1 remapped+peering; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 5.2 KiB/s rd, 391 KiB/s wr, 22 op/s 2024-03-12T02:14:23.293 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:23 smithi153 bash[14340]: audit 2024-03-12T02:14:22.464117+0000 mon.smithi153 (mon.0) 609 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:23.293 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:23 smithi153 bash[14340]: audit 2024-03-12T02:14:22.470858+0000 mon.smithi153 (mon.0) 610 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:23.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:23 smithi163 bash[22347]: audit 2024-03-12T02:14:22.014546+0000 mon.smithi153 (mon.0) 603 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi163", "root=default"]}]': finished 2024-03-12T02:14:23.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:23 smithi163 bash[22347]: cluster 2024-03-12T02:14:22.014659+0000 mon.smithi153 (mon.0) 604 : cluster 0 osdmap e30: 8 total, 5 up, 8 in 2024-03-12T02:14:23.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:23 smithi163 bash[22347]: audit 2024-03-12T02:14:22.014796+0000 mon.smithi153 (mon.0) 605 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:23.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:23 smithi163 bash[22347]: audit 2024-03-12T02:14:22.015313+0000 mon.smithi153 (mon.0) 606 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:23.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:23 smithi163 bash[22347]: audit 2024-03-12T02:14:22.015601+0000 mon.smithi153 (mon.0) 607 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:23.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:23 smithi163 bash[22347]: audit 2024-03-12T02:14:22.019805+0000 mon.smithi153 (mon.0) 608 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:23.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:23 smithi163 bash[22347]: cluster 2024-03-12T02:14:22.335643+0000 mgr.smithi153.asomxr (mgr.14195) 125 : cluster 0 pgmap v89: 1 pgs: 1 remapped+peering; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 5.2 KiB/s rd, 391 KiB/s wr, 22 op/s 2024-03-12T02:14:23.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:23 smithi163 bash[22347]: audit 2024-03-12T02:14:22.464117+0000 mon.smithi153 (mon.0) 609 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:23.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:23 smithi163 bash[22347]: audit 2024-03-12T02:14:22.470858+0000 mon.smithi153 (mon.0) 610 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:24.199 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:14:24.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:24 smithi153 bash[14340]: cluster 2024-03-12T02:14:21.426476+0000 osd.6 (osd.6) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:24.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:24 smithi153 bash[14340]: cluster 2024-03-12T02:14:21.426563+0000 osd.6 (osd.6) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:24.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:24 smithi153 bash[14340]: cluster 2024-03-12T02:14:23.023980+0000 mon.smithi153 (mon.0) 611 : cluster 1 osd.6 [v2:172.21.15.163:6824/1178116342,v1:172.21.15.163:6825/1178116342] boot 2024-03-12T02:14:24.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:24 smithi153 bash[14340]: cluster 2024-03-12T02:14:23.024053+0000 mon.smithi153 (mon.0) 612 : cluster 0 osdmap e31: 8 total, 6 up, 8 in 2024-03-12T02:14:24.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:24 smithi153 bash[14340]: audit 2024-03-12T02:14:23.024365+0000 mon.smithi153 (mon.0) 613 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:24.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:24 smithi153 bash[14340]: audit 2024-03-12T02:14:23.025234+0000 mon.smithi153 (mon.0) 614 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:24.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:24 smithi153 bash[14340]: audit 2024-03-12T02:14:23.025628+0000 mon.smithi153 (mon.0) 615 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:24.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:24 smithi153 bash[14340]: audit 2024-03-12T02:14:23.068889+0000 mon.smithi153 (mon.0) 616 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:24.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:24 smithi153 bash[14340]: audit 2024-03-12T02:14:23.330198+0000 mon.smithi153 (mon.0) 617 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:24.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:24 smithi153 bash[14340]: audit 2024-03-12T02:14:23.337014+0000 mon.smithi153 (mon.0) 618 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:24.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:24 smithi163 bash[22347]: cluster 2024-03-12T02:14:21.426476+0000 osd.6 (osd.6) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:24.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:24 smithi163 bash[22347]: cluster 2024-03-12T02:14:21.426563+0000 osd.6 (osd.6) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:24.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:24 smithi163 bash[22347]: cluster 2024-03-12T02:14:23.023980+0000 mon.smithi153 (mon.0) 611 : cluster 1 osd.6 [v2:172.21.15.163:6824/1178116342,v1:172.21.15.163:6825/1178116342] boot 2024-03-12T02:14:24.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:24 smithi163 bash[22347]: cluster 2024-03-12T02:14:23.024053+0000 mon.smithi153 (mon.0) 612 : cluster 0 osdmap e31: 8 total, 6 up, 8 in 2024-03-12T02:14:24.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:24 smithi163 bash[22347]: audit 2024-03-12T02:14:23.024365+0000 mon.smithi153 (mon.0) 613 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-12T02:14:24.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:24 smithi163 bash[22347]: audit 2024-03-12T02:14:23.025234+0000 mon.smithi153 (mon.0) 614 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:24.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:24 smithi163 bash[22347]: audit 2024-03-12T02:14:23.025628+0000 mon.smithi153 (mon.0) 615 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:24.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:24 smithi163 bash[22347]: audit 2024-03-12T02:14:23.068889+0000 mon.smithi153 (mon.0) 616 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:24.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:24 smithi163 bash[22347]: audit 2024-03-12T02:14:23.330198+0000 mon.smithi153 (mon.0) 617 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:24.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:24 smithi163 bash[22347]: audit 2024-03-12T02:14:23.337014+0000 mon.smithi153 (mon.0) 618 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:25.289 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:25 smithi153 bash[14340]: audit 2024-03-12T02:14:24.021620+0000 mon.smithi153 (mon.0) 619 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:25.289 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:25 smithi153 bash[14340]: audit 2024-03-12T02:14:24.026609+0000 mon.smithi153 (mon.0) 620 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T02:14:25.289 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:25 smithi153 bash[14340]: cluster 2024-03-12T02:14:24.336183+0000 mgr.smithi153.asomxr (mgr.14195) 126 : cluster 0 pgmap v91: 1 pgs: 1 remapped+peering; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 4.1 KiB/s rd, 240 KiB/s wr, 14 op/s 2024-03-12T02:14:25.289 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:25 smithi153 bash[14340]: audit 2024-03-12T02:14:24.944524+0000 mon.smithi153 (mon.0) 621 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T02:14:25.289 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:25 smithi153 bash[14340]: audit 2024-03-12T02:14:24.945449+0000 mon.smithi153 (mon.0) 622 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-12T02:14:25.289 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:25 smithi153 bash[14340]: cluster 2024-03-12T02:14:24.951763+0000 mon.smithi153 (mon.0) 623 : cluster 0 osdmap e32: 8 total, 6 up, 8 in 2024-03-12T02:14:25.289 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:25 smithi153 bash[14340]: audit 2024-03-12T02:14:24.952116+0000 mon.smithi153 (mon.0) 624 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:14:25.289 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:25 smithi153 bash[14340]: audit 2024-03-12T02:14:24.952867+0000 mon.smithi153 (mon.0) 625 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:25.290 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:25 smithi153 bash[14340]: audit 2024-03-12T02:14:24.953095+0000 mon.smithi153 (mon.0) 626 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:25.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:25 smithi163 bash[22347]: audit 2024-03-12T02:14:24.021620+0000 mon.smithi153 (mon.0) 619 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:25.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:25 smithi163 bash[22347]: audit 2024-03-12T02:14:24.026609+0000 mon.smithi153 (mon.0) 620 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T02:14:25.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:25 smithi163 bash[22347]: cluster 2024-03-12T02:14:24.336183+0000 mgr.smithi153.asomxr (mgr.14195) 126 : cluster 0 pgmap v91: 1 pgs: 1 remapped+peering; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 4.1 KiB/s rd, 240 KiB/s wr, 14 op/s 2024-03-12T02:14:25.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:25 smithi163 bash[22347]: audit 2024-03-12T02:14:24.944524+0000 mon.smithi153 (mon.0) 621 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T02:14:25.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:25 smithi163 bash[22347]: audit 2024-03-12T02:14:24.945449+0000 mon.smithi153 (mon.0) 622 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-12T02:14:25.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:25 smithi163 bash[22347]: cluster 2024-03-12T02:14:24.951763+0000 mon.smithi153 (mon.0) 623 : cluster 0 osdmap e32: 8 total, 6 up, 8 in 2024-03-12T02:14:25.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:25 smithi163 bash[22347]: audit 2024-03-12T02:14:24.952116+0000 mon.smithi153 (mon.0) 624 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:14:25.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:25 smithi163 bash[22347]: audit 2024-03-12T02:14:24.952867+0000 mon.smithi153 (mon.0) 625 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:25.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:25 smithi163 bash[22347]: audit 2024-03-12T02:14:24.953095+0000 mon.smithi153 (mon.0) 626 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:27.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:26 smithi153 bash[14340]: audit 2024-03-12T02:14:25.290442+0000 mon.smithi153 (mon.0) 627 : audit 1 from='osd.5 [v2:172.21.15.153:6818/2985927744,v1:172.21.15.153:6819/2985927744]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T02:14:27.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:26 smithi153 bash[14340]: audit 2024-03-12T02:14:25.961552+0000 mon.smithi153 (mon.0) 628 : audit 1 from='osd.5 [v2:172.21.15.153:6818/2985927744,v1:172.21.15.153:6819/2985927744]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-12T02:14:27.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:26 smithi153 bash[14340]: cluster 2024-03-12T02:14:25.961621+0000 mon.smithi153 (mon.0) 629 : cluster 0 osdmap e33: 8 total, 6 up, 8 in 2024-03-12T02:14:27.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:26 smithi153 bash[14340]: audit 2024-03-12T02:14:25.961802+0000 mon.smithi153 (mon.0) 630 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:27.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:26 smithi153 bash[14340]: audit 2024-03-12T02:14:25.962293+0000 mon.smithi153 (mon.0) 631 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:27.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:26 smithi153 bash[14340]: audit 2024-03-12T02:14:25.962555+0000 mon.smithi153 (mon.0) 632 : audit 1 from='osd.5 [v2:172.21.15.153:6818/2985927744,v1:172.21.15.153:6819/2985927744]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-12T02:14:27.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:26 smithi153 bash[14340]: audit 2024-03-12T02:14:25.993205+0000 mon.smithi153 (mon.0) 633 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:27.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:26 smithi153 bash[14340]: audit 2024-03-12T02:14:25.995273+0000 mon.smithi153 (mon.0) 634 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-12T02:14:27.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:26 smithi153 bash[14340]: audit 2024-03-12T02:14:25.995424+0000 mon.smithi153 (mon.0) 635 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:14:27.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:26 smithi163 bash[22347]: audit 2024-03-12T02:14:25.290442+0000 mon.smithi153 (mon.0) 627 : audit 1 from='osd.5 [v2:172.21.15.153:6818/2985927744,v1:172.21.15.153:6819/2985927744]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-12T02:14:27.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:26 smithi163 bash[22347]: audit 2024-03-12T02:14:25.961552+0000 mon.smithi153 (mon.0) 628 : audit 1 from='osd.5 [v2:172.21.15.153:6818/2985927744,v1:172.21.15.153:6819/2985927744]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-12T02:14:27.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:26 smithi163 bash[22347]: cluster 2024-03-12T02:14:25.961621+0000 mon.smithi153 (mon.0) 629 : cluster 0 osdmap e33: 8 total, 6 up, 8 in 2024-03-12T02:14:27.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:26 smithi163 bash[22347]: audit 2024-03-12T02:14:25.961802+0000 mon.smithi153 (mon.0) 630 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:27.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:26 smithi163 bash[22347]: audit 2024-03-12T02:14:25.962293+0000 mon.smithi153 (mon.0) 631 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:27.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:26 smithi163 bash[22347]: audit 2024-03-12T02:14:25.962555+0000 mon.smithi153 (mon.0) 632 : audit 1 from='osd.5 [v2:172.21.15.153:6818/2985927744,v1:172.21.15.153:6819/2985927744]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-12T02:14:27.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:26 smithi163 bash[22347]: audit 2024-03-12T02:14:25.993205+0000 mon.smithi153 (mon.0) 633 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:27.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:26 smithi163 bash[22347]: audit 2024-03-12T02:14:25.995273+0000 mon.smithi153 (mon.0) 634 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-12T02:14:27.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:26 smithi163 bash[22347]: audit 2024-03-12T02:14:25.995424+0000 mon.smithi153 (mon.0) 635 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "mon metadata", "id": "smithi163"}]: dispatch 2024-03-12T02:14:28.138 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:27 smithi153 bash[14340]: audit 2024-03-12T02:14:25.993518+0000 mon.smithi163 (mon.1) 16 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T02:14:28.138 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:27 smithi153 bash[14340]: cluster 2024-03-12T02:14:26.336547+0000 mgr.smithi153.asomxr (mgr.14195) 127 : cluster 0 pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-12T02:14:28.138 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:27 smithi153 bash[14340]: audit 2024-03-12T02:14:26.876348+0000 mon.smithi163 (mon.1) 17 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T02:14:28.138 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:27 smithi153 bash[14340]: audit 2024-03-12T02:14:26.908979+0000 mon.smithi153 (mon.0) 636 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:28.138 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:27 smithi153 bash[14340]: audit 2024-03-12T02:14:26.964201+0000 mon.smithi153 (mon.0) 637 : audit 1 from='osd.5 [v2:172.21.15.153:6818/2985927744,v1:172.21.15.153:6819/2985927744]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-12T02:14:28.138 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:27 smithi153 bash[14340]: cluster 2024-03-12T02:14:26.964323+0000 mon.smithi153 (mon.0) 638 : cluster 0 osdmap e34: 8 total, 6 up, 8 in 2024-03-12T02:14:28.138 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:27 smithi153 bash[14340]: audit 2024-03-12T02:14:26.964560+0000 mon.smithi153 (mon.0) 639 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:28.138 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:27 smithi153 bash[14340]: audit 2024-03-12T02:14:26.965213+0000 mon.smithi153 (mon.0) 640 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:28.138 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:27 smithi153 bash[14340]: audit 2024-03-12T02:14:26.969378+0000 mon.smithi153 (mon.0) 641 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:28.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:27 smithi163 bash[22347]: audit 2024-03-12T02:14:25.993518+0000 mon.smithi163 (mon.1) 16 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-12T02:14:28.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:27 smithi163 bash[22347]: cluster 2024-03-12T02:14:26.336547+0000 mgr.smithi153.asomxr (mgr.14195) 127 : cluster 0 pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-12T02:14:28.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:27 smithi163 bash[22347]: audit 2024-03-12T02:14:26.876348+0000 mon.smithi163 (mon.1) 17 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-12T02:14:28.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:27 smithi163 bash[22347]: audit 2024-03-12T02:14:26.908979+0000 mon.smithi153 (mon.0) 636 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:28.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:27 smithi163 bash[22347]: audit 2024-03-12T02:14:26.964201+0000 mon.smithi153 (mon.0) 637 : audit 1 from='osd.5 [v2:172.21.15.153:6818/2985927744,v1:172.21.15.153:6819/2985927744]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-12T02:14:28.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:27 smithi163 bash[22347]: cluster 2024-03-12T02:14:26.964323+0000 mon.smithi153 (mon.0) 638 : cluster 0 osdmap e34: 8 total, 6 up, 8 in 2024-03-12T02:14:28.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:27 smithi163 bash[22347]: audit 2024-03-12T02:14:26.964560+0000 mon.smithi153 (mon.0) 639 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:28.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:27 smithi163 bash[22347]: audit 2024-03-12T02:14:26.965213+0000 mon.smithi153 (mon.0) 640 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:28.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:27 smithi163 bash[22347]: audit 2024-03-12T02:14:26.969378+0000 mon.smithi153 (mon.0) 641 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:28.384 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:14:29.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:28 smithi153 bash[14340]: cluster 2024-03-12T02:14:26.262821+0000 osd.5 (osd.5) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:29.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:28 smithi153 bash[14340]: cluster 2024-03-12T02:14:26.262878+0000 osd.5 (osd.5) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:29.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:28 smithi153 bash[14340]: audit 2024-03-12T02:14:27.969489+0000 mon.smithi153 (mon.0) 642 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:29.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:28 smithi153 bash[14340]: cluster 2024-03-12T02:14:27.999931+0000 mon.smithi153 (mon.0) 643 : cluster 1 osd.5 [v2:172.21.15.153:6818/2985927744,v1:172.21.15.153:6819/2985927744] boot 2024-03-12T02:14:29.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:28 smithi153 bash[14340]: cluster 2024-03-12T02:14:28.000007+0000 mon.smithi153 (mon.0) 644 : cluster 0 osdmap e35: 8 total, 7 up, 8 in 2024-03-12T02:14:29.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:28 smithi153 bash[14340]: audit 2024-03-12T02:14:28.000567+0000 mon.smithi153 (mon.0) 645 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:29.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:28 smithi153 bash[14340]: audit 2024-03-12T02:14:28.001025+0000 mon.smithi153 (mon.0) 646 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:29.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:28 smithi153 bash[14340]: audit 2024-03-12T02:14:28.381593+0000 mon.smithi153 (mon.0) 647 : audit 0 from='client.? 172.21.15.153:0/2230772096' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:14:29.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:28 smithi163 bash[22347]: cluster 2024-03-12T02:14:26.262821+0000 osd.5 (osd.5) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:29.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:28 smithi163 bash[22347]: cluster 2024-03-12T02:14:26.262878+0000 osd.5 (osd.5) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:29.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:28 smithi163 bash[22347]: audit 2024-03-12T02:14:27.969489+0000 mon.smithi153 (mon.0) 642 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:29.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:28 smithi163 bash[22347]: cluster 2024-03-12T02:14:27.999931+0000 mon.smithi153 (mon.0) 643 : cluster 1 osd.5 [v2:172.21.15.153:6818/2985927744,v1:172.21.15.153:6819/2985927744] boot 2024-03-12T02:14:29.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:28 smithi163 bash[22347]: cluster 2024-03-12T02:14:28.000007+0000 mon.smithi153 (mon.0) 644 : cluster 0 osdmap e35: 8 total, 7 up, 8 in 2024-03-12T02:14:29.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:28 smithi163 bash[22347]: audit 2024-03-12T02:14:28.000567+0000 mon.smithi153 (mon.0) 645 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-12T02:14:29.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:28 smithi163 bash[22347]: audit 2024-03-12T02:14:28.001025+0000 mon.smithi153 (mon.0) 646 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:29.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:28 smithi163 bash[22347]: audit 2024-03-12T02:14:28.381593+0000 mon.smithi153 (mon.0) 647 : audit 0 from='client.? 172.21.15.153:0/2230772096' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:14:29.800 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":35,"num_osds":8,"num_up_osds":7,"osd_up_since":1710209667,"num_in_osds":8,"osd_in_since":1710209619,"num_remapped_pgs":0} 2024-03-12T02:14:30.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:29 smithi153 bash[14340]: cluster 2024-03-12T02:14:28.337104+0000 mgr.smithi153.asomxr (mgr.14195) 128 : cluster 0 pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 522 KiB/s wr, 31 op/s; 144 KiB/s, 0 objects/s recovering 2024-03-12T02:14:30.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:29 smithi153 bash[14340]: cluster 2024-03-12T02:14:29.005881+0000 mon.smithi153 (mon.0) 648 : cluster 0 osdmap e36: 8 total, 7 up, 8 in 2024-03-12T02:14:30.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:29 smithi153 bash[14340]: audit 2024-03-12T02:14:29.006011+0000 mon.smithi153 (mon.0) 649 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:30.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:29 smithi163 bash[22347]: cluster 2024-03-12T02:14:28.337104+0000 mgr.smithi153.asomxr (mgr.14195) 128 : cluster 0 pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 522 KiB/s wr, 31 op/s; 144 KiB/s, 0 objects/s recovering 2024-03-12T02:14:30.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:29 smithi163 bash[22347]: cluster 2024-03-12T02:14:29.005881+0000 mon.smithi153 (mon.0) 648 : cluster 0 osdmap e36: 8 total, 7 up, 8 in 2024-03-12T02:14:30.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:29 smithi163 bash[22347]: audit 2024-03-12T02:14:29.006011+0000 mon.smithi153 (mon.0) 649 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:30.801 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-12T02:14:31.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:30 smithi153 bash[14340]: audit 2024-03-12T02:14:30.369855+0000 mon.smithi153 (mon.0) 650 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:14:31.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:30 smithi163 bash[22347]: audit 2024-03-12T02:14:30.369855+0000 mon.smithi153 (mon.0) 650 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:14:32.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:31 smithi153 bash[14340]: cluster 2024-03-12T02:14:30.337674+0000 mgr.smithi153.asomxr (mgr.14195) 129 : cluster 0 pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 8.2 KiB/s rd, 477 KiB/s wr, 29 op/s; 132 KiB/s, 0 objects/s recovering 2024-03-12T02:14:32.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:31 smithi153 bash[14340]: audit 2024-03-12T02:14:31.782480+0000 mon.smithi153 (mon.0) 651 : audit 1 from='osd.7 [v2:172.21.15.153:6826/3471588101,v1:172.21.15.153:6827/3471588101]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-12T02:14:32.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:31 smithi163 bash[22347]: cluster 2024-03-12T02:14:30.337674+0000 mgr.smithi153.asomxr (mgr.14195) 129 : cluster 0 pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 8.2 KiB/s rd, 477 KiB/s wr, 29 op/s; 132 KiB/s, 0 objects/s recovering 2024-03-12T02:14:32.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:31 smithi163 bash[22347]: audit 2024-03-12T02:14:31.782480+0000 mon.smithi153 (mon.0) 651 : audit 1 from='osd.7 [v2:172.21.15.153:6826/3471588101,v1:172.21.15.153:6827/3471588101]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-12T02:14:33.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:32 smithi163 bash[22347]: audit 2024-03-12T02:14:31.906078+0000 mon.smithi153 (mon.0) 652 : audit 1 from='osd.7 [v2:172.21.15.153:6826/3471588101,v1:172.21.15.153:6827/3471588101]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-12T02:14:33.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:32 smithi163 bash[22347]: cluster 2024-03-12T02:14:31.906147+0000 mon.smithi153 (mon.0) 653 : cluster 0 osdmap e37: 8 total, 7 up, 8 in 2024-03-12T02:14:33.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:32 smithi163 bash[22347]: audit 2024-03-12T02:14:31.906332+0000 mon.smithi153 (mon.0) 654 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:33.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:32 smithi163 bash[22347]: audit 2024-03-12T02:14:31.907074+0000 mon.smithi153 (mon.0) 655 : audit 1 from='osd.7 [v2:172.21.15.153:6826/3471588101,v1:172.21.15.153:6827/3471588101]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-12T02:14:33.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:32 smithi163 bash[22347]: cluster 2024-03-12T02:14:32.338238+0000 mgr.smithi153.asomxr (mgr.14195) 130 : cluster 0 pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 1.7 KiB/s rd, 49 KiB/s wr, 4 op/s 2024-03-12T02:14:33.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:32 smithi153 bash[14340]: audit 2024-03-12T02:14:31.906078+0000 mon.smithi153 (mon.0) 652 : audit 1 from='osd.7 [v2:172.21.15.153:6826/3471588101,v1:172.21.15.153:6827/3471588101]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-12T02:14:33.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:32 smithi153 bash[14340]: cluster 2024-03-12T02:14:31.906147+0000 mon.smithi153 (mon.0) 653 : cluster 0 osdmap e37: 8 total, 7 up, 8 in 2024-03-12T02:14:33.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:32 smithi153 bash[14340]: audit 2024-03-12T02:14:31.906332+0000 mon.smithi153 (mon.0) 654 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:33.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:32 smithi153 bash[14340]: audit 2024-03-12T02:14:31.907074+0000 mon.smithi153 (mon.0) 655 : audit 1 from='osd.7 [v2:172.21.15.153:6826/3471588101,v1:172.21.15.153:6827/3471588101]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-12T02:14:33.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:32 smithi153 bash[14340]: cluster 2024-03-12T02:14:32.338238+0000 mgr.smithi153.asomxr (mgr.14195) 130 : cluster 0 pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 1.7 KiB/s rd, 49 KiB/s wr, 4 op/s 2024-03-12T02:14:34.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:33 smithi163 bash[22347]: audit 2024-03-12T02:14:32.909352+0000 mon.smithi153 (mon.0) 656 : audit 1 from='osd.7 [v2:172.21.15.153:6826/3471588101,v1:172.21.15.153:6827/3471588101]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-12T02:14:34.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:33 smithi163 bash[22347]: cluster 2024-03-12T02:14:32.909456+0000 mon.smithi153 (mon.0) 657 : cluster 0 osdmap e38: 8 total, 7 up, 8 in 2024-03-12T02:14:34.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:33 smithi163 bash[22347]: audit 2024-03-12T02:14:32.909983+0000 mon.smithi153 (mon.0) 658 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:34.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:33 smithi163 bash[22347]: audit 2024-03-12T02:14:32.915967+0000 mon.smithi153 (mon.0) 659 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:34.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:33 smithi163 bash[22347]: audit 2024-03-12T02:14:33.664774+0000 mon.smithi153 (mon.0) 660 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:34.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:33 smithi163 bash[22347]: audit 2024-03-12T02:14:33.675043+0000 mon.smithi153 (mon.0) 661 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:34.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:33 smithi163 bash[22347]: audit 2024-03-12T02:14:33.756623+0000 mon.smithi153 (mon.0) 662 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:14:34.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:33 smithi153 bash[14340]: audit 2024-03-12T02:14:32.909352+0000 mon.smithi153 (mon.0) 656 : audit 1 from='osd.7 [v2:172.21.15.153:6826/3471588101,v1:172.21.15.153:6827/3471588101]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-12T02:14:34.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:33 smithi153 bash[14340]: cluster 2024-03-12T02:14:32.909456+0000 mon.smithi153 (mon.0) 657 : cluster 0 osdmap e38: 8 total, 7 up, 8 in 2024-03-12T02:14:34.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:33 smithi153 bash[14340]: audit 2024-03-12T02:14:32.909983+0000 mon.smithi153 (mon.0) 658 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:34.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:33 smithi153 bash[14340]: audit 2024-03-12T02:14:32.915967+0000 mon.smithi153 (mon.0) 659 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:34.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:33 smithi153 bash[14340]: audit 2024-03-12T02:14:33.664774+0000 mon.smithi153 (mon.0) 660 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:34.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:33 smithi153 bash[14340]: audit 2024-03-12T02:14:33.675043+0000 mon.smithi153 (mon.0) 661 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:34.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:33 smithi153 bash[14340]: audit 2024-03-12T02:14:33.756623+0000 mon.smithi153 (mon.0) 662 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:14:35.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:34 smithi163 bash[22347]: cluster 2024-03-12T02:14:32.729292+0000 osd.7 (osd.7) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:35.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:34 smithi163 bash[22347]: cluster 2024-03-12T02:14:32.729445+0000 osd.7 (osd.7) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:35.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:34 smithi163 bash[22347]: audit 2024-03-12T02:14:33.919165+0000 mon.smithi153 (mon.0) 663 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:35.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:34 smithi163 bash[22347]: cluster 2024-03-12T02:14:33.924142+0000 mon.smithi153 (mon.0) 664 : cluster 1 osd.7 [v2:172.21.15.153:6826/3471588101,v1:172.21.15.153:6827/3471588101] boot 2024-03-12T02:14:35.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:34 smithi163 bash[22347]: cluster 2024-03-12T02:14:33.924201+0000 mon.smithi153 (mon.0) 665 : cluster 0 osdmap e39: 8 total, 8 up, 8 in 2024-03-12T02:14:35.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:34 smithi163 bash[22347]: audit 2024-03-12T02:14:33.924683+0000 mon.smithi153 (mon.0) 666 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:35.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:34 smithi163 bash[22347]: cluster 2024-03-12T02:14:34.338870+0000 mgr.smithi153.asomxr (mgr.14195) 131 : cluster 0 pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 1.7 KiB/s rd, 49 KiB/s wr, 4 op/s 2024-03-12T02:14:35.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:34 smithi153 bash[14340]: cluster 2024-03-12T02:14:32.729292+0000 osd.7 (osd.7) 1 : cluster 0 purged_snaps scrub starts 2024-03-12T02:14:35.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:34 smithi153 bash[14340]: cluster 2024-03-12T02:14:32.729445+0000 osd.7 (osd.7) 2 : cluster 0 purged_snaps scrub ok 2024-03-12T02:14:35.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:34 smithi153 bash[14340]: audit 2024-03-12T02:14:33.919165+0000 mon.smithi153 (mon.0) 663 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:35.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:34 smithi153 bash[14340]: cluster 2024-03-12T02:14:33.924142+0000 mon.smithi153 (mon.0) 664 : cluster 1 osd.7 [v2:172.21.15.153:6826/3471588101,v1:172.21.15.153:6827/3471588101] boot 2024-03-12T02:14:35.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:34 smithi153 bash[14340]: cluster 2024-03-12T02:14:33.924201+0000 mon.smithi153 (mon.0) 665 : cluster 0 osdmap e39: 8 total, 8 up, 8 in 2024-03-12T02:14:35.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:34 smithi153 bash[14340]: audit 2024-03-12T02:14:33.924683+0000 mon.smithi153 (mon.0) 666 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-12T02:14:35.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:34 smithi153 bash[14340]: cluster 2024-03-12T02:14:34.338870+0000 mgr.smithi153.asomxr (mgr.14195) 131 : cluster 0 pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail; 1.7 KiB/s rd, 49 KiB/s wr, 4 op/s 2024-03-12T02:14:36.167 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:14:36.196 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:35 smithi153 bash[14340]: cluster 2024-03-12T02:14:34.925540+0000 mon.smithi153 (mon.0) 667 : cluster 0 osdmap e40: 8 total, 8 up, 8 in 2024-03-12T02:14:36.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:35 smithi163 bash[22347]: cluster 2024-03-12T02:14:34.925540+0000 mon.smithi153 (mon.0) 667 : cluster 0 osdmap e40: 8 total, 8 up, 8 in 2024-03-12T02:14:37.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:36 smithi163 bash[22347]: cluster 2024-03-12T02:14:35.934671+0000 mon.smithi153 (mon.0) 668 : cluster 0 osdmap e41: 8 total, 8 up, 8 in 2024-03-12T02:14:37.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:36 smithi163 bash[22347]: cluster 2024-03-12T02:14:36.339377+0000 mgr.smithi153.asomxr (mgr.14195) 132 : cluster 0 pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:14:37.280 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:36 smithi153 bash[14340]: cluster 2024-03-12T02:14:35.934671+0000 mon.smithi153 (mon.0) 668 : cluster 0 osdmap e41: 8 total, 8 up, 8 in 2024-03-12T02:14:37.280 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:36 smithi153 bash[14340]: cluster 2024-03-12T02:14:36.339377+0000 mgr.smithi153.asomxr (mgr.14195) 132 : cluster 0 pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:14:38.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:37 smithi163 bash[22347]: cluster 2024-03-12T02:14:36.945076+0000 mon.smithi153 (mon.0) 669 : cluster 0 osdmap e42: 8 total, 8 up, 8 in 2024-03-12T02:14:38.268 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:37 smithi153 bash[14340]: cluster 2024-03-12T02:14:36.945076+0000 mon.smithi153 (mon.0) 669 : cluster 0 osdmap e42: 8 total, 8 up, 8 in 2024-03-12T02:14:39.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:38 smithi163 bash[22347]: cluster 2024-03-12T02:14:37.945352+0000 mon.smithi153 (mon.0) 670 : cluster 0 osdmap e43: 8 total, 8 up, 8 in 2024-03-12T02:14:39.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:38 smithi163 bash[22347]: cluster 2024-03-12T02:14:38.339827+0000 mgr.smithi153.asomxr (mgr.14195) 133 : cluster 0 pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:14:39.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:38 smithi153 bash[14340]: cluster 2024-03-12T02:14:37.945352+0000 mon.smithi153 (mon.0) 670 : cluster 0 osdmap e43: 8 total, 8 up, 8 in 2024-03-12T02:14:39.321 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:38 smithi153 bash[14340]: cluster 2024-03-12T02:14:38.339827+0000 mgr.smithi153.asomxr (mgr.14195) 133 : cluster 0 pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:14:39.590 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:14:40.380 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":43,"num_osds":8,"num_up_osds":8,"osd_up_since":1710209673,"num_in_osds":8,"osd_in_since":1710209619,"num_remapped_pgs":0} 2024-03-12T02:14:40.381 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-12T02:14:40.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:40 smithi163 bash[22347]: audit 2024-03-12T02:14:39.404887+0000 mon.smithi153 (mon.0) 671 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:40.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:40 smithi163 bash[22347]: audit 2024-03-12T02:14:39.414043+0000 mon.smithi153 (mon.0) 672 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:40.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:40 smithi163 bash[22347]: audit 2024-03-12T02:14:39.588352+0000 mon.smithi153 (mon.0) 673 : audit 0 from='client.? 172.21.15.153:0/686278683' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:14:40.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:40 smithi163 bash[22347]: audit 2024-03-12T02:14:39.966706+0000 mon.smithi153 (mon.0) 674 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:40.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:40 smithi163 bash[22347]: audit 2024-03-12T02:14:39.973158+0000 mon.smithi153 (mon.0) 675 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:40 smithi153 bash[14340]: audit 2024-03-12T02:14:39.404887+0000 mon.smithi153 (mon.0) 671 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:40 smithi153 bash[14340]: audit 2024-03-12T02:14:39.414043+0000 mon.smithi153 (mon.0) 672 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:40 smithi153 bash[14340]: audit 2024-03-12T02:14:39.588352+0000 mon.smithi153 (mon.0) 673 : audit 0 from='client.? 172.21.15.153:0/686278683' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-12T02:14:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:40 smithi153 bash[14340]: audit 2024-03-12T02:14:39.966706+0000 mon.smithi153 (mon.0) 674 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:40.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:40 smithi153 bash[14340]: audit 2024-03-12T02:14:39.973158+0000 mon.smithi153 (mon.0) 675 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:41.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:41 smithi163 bash[22347]: cluster 2024-03-12T02:14:40.340231+0000 mgr.smithi153.asomxr (mgr.14195) 134 : cluster 0 pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-03-12T02:14:41.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:41 smithi153 bash[14340]: cluster 2024-03-12T02:14:40.340231+0000 mgr.smithi153.asomxr (mgr.14195) 134 : cluster 0 pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-03-12T02:14:43.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:43 smithi163 bash[22347]: cluster 2024-03-12T02:14:42.340750+0000 mgr.smithi153.asomxr (mgr.14195) 135 : cluster 0 pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-12T02:14:44.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:43 smithi153 bash[14340]: cluster 2024-03-12T02:14:42.340750+0000 mgr.smithi153.asomxr (mgr.14195) 135 : cluster 0 pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-12T02:14:45.123 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:14:45.934 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:45 smithi163 bash[22347]: cluster 2024-03-12T02:14:44.341490+0000 mgr.smithi153.asomxr (mgr.14195) 136 : cluster 0 pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-12T02:14:45.934 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:45 smithi163 bash[22347]: audit 2024-03-12T02:14:45.370543+0000 mon.smithi153 (mon.0) 676 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:14:46.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:45 smithi153 bash[14340]: cluster 2024-03-12T02:14:44.341490+0000 mgr.smithi153.asomxr (mgr.14195) 136 : cluster 0 pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-12T02:14:46.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:45 smithi153 bash[14340]: audit 2024-03-12T02:14:45.370543+0000 mon.smithi153 (mon.0) 676 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:14:47.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:47 smithi163 bash[22347]: cluster 2024-03-12T02:14:46.342212+0000 mgr.smithi153.asomxr (mgr.14195) 137 : cluster 0 pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-12T02:14:48.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:47 smithi153 bash[14340]: cluster 2024-03-12T02:14:46.342212+0000 mgr.smithi153.asomxr (mgr.14195) 137 : cluster 0 pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-12T02:14:49.360 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:14:49.360 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":43,"fsid":"5cbf0f4e-e015-11ee-95c8-87774f69a715","created":"2024-03-12T02:09:06.107943+0000","modified":"2024-03-12T02:14:37.939127+0000","last_up_change":"2024-03-12T02:14:33.911715+0000","last_in_change":"2024-03-12T02:13:39.749392+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":17,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T02:14:14.892829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad1d33d5-2e6b-4e4b-b148-3cb45de94e09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6801","nonce":3954737342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6803","nonce":3954737342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6807","nonce":3954737342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6805","nonce":3954737342}]},"public_addr":"172.21.15.163:6801/3954737342","cluster_addr":"172.21.15.163:6803/3954737342","heartbeat_back_addr":"172.21.15.163:6807/3954737342","heartbeat_front_addr":"172.21.15.163:6805/3954737342","state":["exists","up"]},{"osd":1,"uuid":"e789a733-1bcc-4a47-8c1f-6190d10a0dba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6803","nonce":799977908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6805","nonce":799977908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6809","nonce":799977908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6807","nonce":799977908}]},"public_addr":"172.21.15.153:6803/799977908","cluster_addr":"172.21.15.153:6805/799977908","heartbeat_back_addr":"172.21.15.153:6809/799977908","heartbeat_front_addr":"172.21.15.153:6807/799977908","state":["exists","up"]},{"osd":2,"uuid":"d17e4690-4fa9-45b1-8fe6-e62f6b2475ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6809","nonce":2994034967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6811","nonce":2994034967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6815","nonce":2994034967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6813","nonce":2994034967}]},"public_addr":"172.21.15.163:6809/2994034967","cluster_addr":"172.21.15.163:6811/2994034967","heartbeat_back_addr":"172.21.15.163:6815/2994034967","heartbeat_front_addr":"172.21.15.163:6813/2994034967","state":["exists","up"]},{"osd":3,"uuid":"fc7e829f-69ae-482f-9633-ba76231f39a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6811","nonce":347906783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6813","nonce":347906783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6817","nonce":347906783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6815","nonce":347906783}]},"public_addr":"172.21.15.153:6811/347906783","cluster_addr":"172.21.15.153:6813/347906783","heartbeat_back_addr":"172.21.15.153:6817/347906783","heartbeat_front_addr":"172.21.15.153:6815/347906783","state":["exists","up"]},{"osd":4,"uuid":"40f0bafb-ca9a-47ca-9ec2-1d03004e6efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6817","nonce":1978911710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6819","nonce":1978911710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6823","nonce":1978911710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6821","nonce":1978911710}]},"public_addr":"172.21.15.163:6817/1978911710","cluster_addr":"172.21.15.163:6819/1978911710","heartbeat_back_addr":"172.21.15.163:6823/1978911710","heartbeat_front_addr":"172.21.15.163:6821/1978911710","state":["exists","up"]},{"osd":5,"uuid":"301fd961-da29-4b85-8683-c117bb4a5086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6819","nonce":2985927744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6821","nonce":2985927744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6825","nonce":2985927744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6823","nonce":2985927744}]},"public_addr":"172.21.15.153:6819/2985927744","cluster_addr":"172.21.15.153:6821/2985927744","heartbeat_back_addr":"172.21.15.153:6825/2985927744","heartbeat_front_addr":"172.21.15.153:6823/2985927744","state":["exists","up"]},{"osd":6,"uuid":"e19436c1-f7ac-415b-b4be-1c2c52c87473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6825","nonce":1178116342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6827","nonce":1178116342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6831","nonce":1178116342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6829","nonce":1178116342}]},"public_addr":"172.21.15.163:6825/1178116342","cluster_addr":"172.21.15.163:6827/1178116342","heartbeat_back_addr":"172.21.15.163:6831/1178116342","heartbeat_front_addr":"172.21.15.163:6829/1178116342","state":["exists","up"]},{"osd":7,"uuid":"a76fe6b3-b2ae-4fab-b922-fd436208b6db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6827","nonce":3471588101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6829","nonce":3471588101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6833","nonce":3471588101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6831","nonce":3471588101}]},"public_addr":"172.21.15.153:6827/3471588101","cluster_addr":"172.21.15.153:6829/3471588101","heartbeat_back_addr":"172.21.15.153:6833/3471588101","heartbeat_front_addr":"172.21.15.153:6831/3471588101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:05.021168+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:10.860357+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:11.204898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:19.155151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:16.681764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:26.262881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:21.426567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:32.729454+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.153:6801/689479112":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/4010545582":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/3030920440":"2024-03-13T02:10:29.011431+0000","172.21.15.153:6801/1133623500":"2024-03-13T02:10:29.011431+0000","172.21.15.153:0/3201040853":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/178648720":"2024-03-13T02:10:29.011431+0000","172.21.15.153:0/1242805769":"2024-03-13T02:10:29.011431+0000","172.21.15.153:6800/689479112":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/1363796892":"2024-03-13T02:09:40.347470+0000","172.21.15.153:0/714144036":"2024-03-13T02:09:40.347470+0000","172.21.15.153:0/4185653963":"2024-03-13T02:09:40.347470+0000","172.21.15.153:0/2183141425":"2024-03-13T02:12:00.299133+0000","172.21.15.153:6800/296961988":"2024-03-13T02:09:40.347470+0000","172.21.15.153:6800/1133623500":"2024-03-13T02:10:29.011431+0000","172.21.15.153:6801/296961988":"2024-03-13T02:09:40.347470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T02:14:49.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:49 smithi153 bash[14340]: cluster 2024-03-12T02:14:48.342887+0000 mgr.smithi153.asomxr (mgr.14195) 138 : cluster 0 pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-12T02:14:49.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:49 smithi153 bash[14340]: audit 2024-03-12T02:14:49.355730+0000 mon.smithi153 (mon.0) 677 : audit 0 from='client.? 172.21.15.153:0/429473679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T02:14:49.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:49 smithi163 bash[22347]: cluster 2024-03-12T02:14:48.342887+0000 mgr.smithi153.asomxr (mgr.14195) 138 : cluster 0 pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-12T02:14:49.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:49 smithi163 bash[22347]: audit 2024-03-12T02:14:49.355730+0000 mon.smithi153 (mon.0) 677 : audit 0 from='client.? 172.21.15.153:0/429473679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T02:14:50.213 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-12T02:14:14.892829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-12T02:14:50.213 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-12T02:14:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:51 smithi163 bash[22347]: cluster 2024-03-12T02:14:50.343295+0000 mgr.smithi153.asomxr (mgr.14195) 139 : cluster 0 pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-12T02:14:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:51 smithi163 bash[22347]: cephadm 2024-03-12T02:14:50.681670+0000 mgr.smithi153.asomxr (mgr.14195) 140 : cephadm 1 Detected new or changed devices on smithi163 2024-03-12T02:14:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:51 smithi163 bash[22347]: audit 2024-03-12T02:14:50.690496+0000 mon.smithi153 (mon.0) 678 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:51 smithi163 bash[22347]: audit 2024-03-12T02:14:50.700204+0000 mon.smithi153 (mon.0) 679 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:51 smithi163 bash[22347]: audit 2024-03-12T02:14:50.702193+0000 mon.smithi153 (mon.0) 680 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:51 smithi163 bash[22347]: audit 2024-03-12T02:14:50.703468+0000 mon.smithi153 (mon.0) 681 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:51 smithi163 bash[22347]: audit 2024-03-12T02:14:50.704637+0000 mon.smithi153 (mon.0) 682 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:51 smithi163 bash[22347]: audit 2024-03-12T02:14:50.705968+0000 mon.smithi153 (mon.0) 683 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:51 smithi163 bash[22347]: cephadm 2024-03-12T02:14:50.706842+0000 mgr.smithi153.asomxr (mgr.14195) 141 : cephadm 1 Adjusting osd_memory_target on smithi163 to 3772M 2024-03-12T02:14:51.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:51 smithi163 bash[22347]: audit 2024-03-12T02:14:50.713311+0000 mon.smithi153 (mon.0) 684 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:52.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:51 smithi153 bash[14340]: cluster 2024-03-12T02:14:50.343295+0000 mgr.smithi153.asomxr (mgr.14195) 139 : cluster 0 pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-12T02:14:52.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:51 smithi153 bash[14340]: cephadm 2024-03-12T02:14:50.681670+0000 mgr.smithi153.asomxr (mgr.14195) 140 : cephadm 1 Detected new or changed devices on smithi163 2024-03-12T02:14:52.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:51 smithi153 bash[14340]: audit 2024-03-12T02:14:50.690496+0000 mon.smithi153 (mon.0) 678 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:52.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:51 smithi153 bash[14340]: audit 2024-03-12T02:14:50.700204+0000 mon.smithi153 (mon.0) 679 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:52.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:51 smithi153 bash[14340]: audit 2024-03-12T02:14:50.702193+0000 mon.smithi153 (mon.0) 680 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:52.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:51 smithi153 bash[14340]: audit 2024-03-12T02:14:50.703468+0000 mon.smithi153 (mon.0) 681 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:52.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:51 smithi153 bash[14340]: audit 2024-03-12T02:14:50.704637+0000 mon.smithi153 (mon.0) 682 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:52.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:51 smithi153 bash[14340]: audit 2024-03-12T02:14:50.705968+0000 mon.smithi153 (mon.0) 683 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:52.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:51 smithi153 bash[14340]: cephadm 2024-03-12T02:14:50.706842+0000 mgr.smithi153.asomxr (mgr.14195) 141 : cephadm 1 Adjusting osd_memory_target on smithi163 to 3772M 2024-03-12T02:14:52.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:51 smithi153 bash[14340]: audit 2024-03-12T02:14:50.713311+0000 mon.smithi153 (mon.0) 684 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:53.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: cluster 2024-03-12T02:14:52.343831+0000 mgr.smithi153.asomxr (mgr.14195) 142 : cluster 0 pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:14:53.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:52.954283+0000 mon.smithi153 (mon.0) 685 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:53.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:52.963765+0000 mon.smithi153 (mon.0) 686 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:53.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:52.965818+0000 mon.smithi153 (mon.0) 687 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:53.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:52.967048+0000 mon.smithi153 (mon.0) 688 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:53.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:52.968246+0000 mon.smithi153 (mon.0) 689 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:52.969432+0000 mon.smithi153 (mon.0) 690 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:52.976989+0000 mon.smithi153 (mon.0) 691 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:52.980327+0000 mon.smithi153 (mon.0) 692 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:52.981560+0000 mon.smithi153 (mon.0) 693 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:14:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:52.988714+0000 mon.smithi153 (mon.0) 694 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:52.994304+0000 mon.smithi153 (mon.0) 695 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:14:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:53.001791+0000 mon.smithi153 (mon.0) 696 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T02:14:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:53.003049+0000 mon.smithi153 (mon.0) 697 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:53.007675+0000 mon.smithi153 (mon.0) 698 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T02:14:53.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:53 smithi163 bash[22347]: audit 2024-03-12T02:14:53.008868+0000 mon.smithi153 (mon.0) 699 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:54.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: cluster 2024-03-12T02:14:52.343831+0000 mgr.smithi153.asomxr (mgr.14195) 142 : cluster 0 pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:14:54.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:52.954283+0000 mon.smithi153 (mon.0) 685 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:54.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:52.963765+0000 mon.smithi153 (mon.0) 686 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:54.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:52.965818+0000 mon.smithi153 (mon.0) 687 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:54.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:52.967048+0000 mon.smithi153 (mon.0) 688 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:54.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:52.968246+0000 mon.smithi153 (mon.0) 689 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:54.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:52.969432+0000 mon.smithi153 (mon.0) 690 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:14:54.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:52.976989+0000 mon.smithi153 (mon.0) 691 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:54.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:52.980327+0000 mon.smithi153 (mon.0) 692 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:54.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:52.981560+0000 mon.smithi153 (mon.0) 693 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:14:54.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:52.988714+0000 mon.smithi153 (mon.0) 694 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:14:54.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:52.994304+0000 mon.smithi153 (mon.0) 695 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:14:54.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:53.001791+0000 mon.smithi153 (mon.0) 696 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T02:14:54.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:53.003049+0000 mon.smithi153 (mon.0) 697 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:54.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:53.007675+0000 mon.smithi153 (mon.0) 698 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-12T02:14:54.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:53 smithi153 bash[14340]: audit 2024-03-12T02:14:53.008868+0000 mon.smithi153 (mon.0) 699 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:14:54.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:54 smithi163 bash[22347]: cephadm 2024-03-12T02:14:52.943467+0000 mgr.smithi153.asomxr (mgr.14195) 143 : cephadm 1 Detected new or changed devices on smithi153 2024-03-12T02:14:54.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:54 smithi163 bash[22347]: cephadm 2024-03-12T02:14:52.970316+0000 mgr.smithi153.asomxr (mgr.14195) 144 : cephadm 1 Adjusting osd_memory_target on smithi153 to 3004M 2024-03-12T02:14:55.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:54 smithi153 bash[14340]: cephadm 2024-03-12T02:14:52.943467+0000 mgr.smithi153.asomxr (mgr.14195) 143 : cephadm 1 Detected new or changed devices on smithi153 2024-03-12T02:14:55.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:54 smithi153 bash[14340]: cephadm 2024-03-12T02:14:52.970316+0000 mgr.smithi153.asomxr (mgr.14195) 144 : cephadm 1 Adjusting osd_memory_target on smithi153 to 3004M 2024-03-12T02:14:55.456 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:14:55.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:55 smithi153 bash[14340]: cluster 2024-03-12T02:14:54.344490+0000 mgr.smithi153.asomxr (mgr.14195) 145 : cluster 0 pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:14:55.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:55 smithi163 bash[22347]: cluster 2024-03-12T02:14:54.344490+0000 mgr.smithi153.asomxr (mgr.14195) 145 : cluster 0 pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:14:57.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:57 smithi163 bash[22347]: cluster 2024-03-12T02:14:56.345156+0000 mgr.smithi153.asomxr (mgr.14195) 146 : cluster 0 pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:14:58.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:57 smithi153 bash[14340]: cluster 2024-03-12T02:14:56.345156+0000 mgr.smithi153.asomxr (mgr.14195) 146 : cluster 0 pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:14:59.300 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2024-03-12T02:14:59.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:59 smithi163 bash[22347]: cluster 2024-03-12T02:14:58.345738+0000 mgr.smithi153.asomxr (mgr.14195) 147 : cluster 0 pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:14:59.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:14:59 smithi163 bash[22347]: audit 2024-03-12T02:14:59.297774+0000 mon.smithi153 (mon.0) 700 : audit 0 from='client.? 172.21.15.153:0/1562205500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T02:15:00.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:59 smithi153 bash[14340]: cluster 2024-03-12T02:14:58.345738+0000 mgr.smithi153.asomxr (mgr.14195) 147 : cluster 0 pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:00.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:14:59 smithi153 bash[14340]: audit 2024-03-12T02:14:59.297774+0000 mon.smithi153 (mon.0) 700 : audit 0 from='client.? 172.21.15.153:0/1562205500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-12T02:15:00.536 INFO:tasks.cephadm:Setting up client nodes... 2024-03-12T02:15:00.536 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-12T02:15:00.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:00 smithi153 bash[14340]: audit 2024-03-12T02:15:00.374807+0000 mon.smithi153 (mon.0) 701 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:15:00.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:00 smithi163 bash[22347]: audit 2024-03-12T02:15:00.374807+0000 mon.smithi153 (mon.0) 701 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:15:01.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:01 smithi153 bash[14340]: cluster 2024-03-12T02:15:00.346326+0000 mgr.smithi153.asomxr (mgr.14195) 148 : cluster 0 pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:01.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:01 smithi163 bash[22347]: cluster 2024-03-12T02:15:00.346326+0000 mgr.smithi153.asomxr (mgr.14195) 148 : cluster 0 pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:03.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:03 smithi153 bash[14340]: cluster 2024-03-12T02:15:02.346942+0000 mgr.smithi153.asomxr (mgr.14195) 149 : cluster 0 pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:03.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:03 smithi163 bash[22347]: cluster 2024-03-12T02:15:02.346942+0000 mgr.smithi153.asomxr (mgr.14195) 149 : cluster 0 pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:05.268 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:05.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:05 smithi163 bash[22347]: cluster 2024-03-12T02:15:04.347389+0000 mgr.smithi153.asomxr (mgr.14195) 150 : cluster 0 pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:06.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:05 smithi153 bash[14340]: cluster 2024-03-12T02:15:04.347389+0000 mgr.smithi153.asomxr (mgr.14195) 150 : cluster 0 pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:07.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:07 smithi163 bash[22347]: cluster 2024-03-12T02:15:06.347898+0000 mgr.smithi153.asomxr (mgr.14195) 151 : cluster 0 pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:08.017 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:07 smithi153 bash[14340]: cluster 2024-03-12T02:15:06.347898+0000 mgr.smithi153.asomxr (mgr.14195) 151 : cluster 0 pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:08.514 INFO:teuthology.orchestra.run.smithi153.stdout:[client.0] 2024-03-12T02:15:08.515 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCsuu9lF1g6HhAAQdSrHtrPZvhk2JlSLU/3Qw== 2024-03-12T02:15:08.929 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:08 smithi163 bash[22347]: audit 2024-03-12T02:15:08.506790+0000 mon.smithi153 (mon.0) 702 : audit 1 from='client.? 172.21.15.153:0/3626222799' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T02:15:08.930 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:08 smithi163 bash[22347]: audit 2024-03-12T02:15:08.511900+0000 mon.smithi153 (mon.0) 703 : audit 1 from='client.? 172.21.15.153:0/3626222799' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-12T02:15:09.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:08 smithi153 bash[14340]: audit 2024-03-12T02:15:08.506790+0000 mon.smithi153 (mon.0) 702 : audit 1 from='client.? 172.21.15.153:0/3626222799' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T02:15:09.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:08 smithi153 bash[14340]: audit 2024-03-12T02:15:08.511900+0000 mon.smithi153 (mon.0) 703 : audit 1 from='client.? 172.21.15.153:0/3626222799' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-12T02:15:09.692 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T02:15:09.692 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-12T02:15:09.692 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-12T02:15:09.713 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-12T02:15:09.969 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:09 smithi153 bash[14340]: cluster 2024-03-12T02:15:08.348354+0000 mgr.smithi153.asomxr (mgr.14195) 152 : cluster 0 pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:09.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:09 smithi163 bash[22347]: cluster 2024-03-12T02:15:08.348354+0000 mgr.smithi153.asomxr (mgr.14195) 152 : cluster 0 pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:12.022 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:11 smithi153 bash[14340]: cluster 2024-03-12T02:15:10.348881+0000 mgr.smithi153.asomxr (mgr.14195) 153 : cluster 0 pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:12.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:11 smithi163 bash[22347]: cluster 2024-03-12T02:15:10.348881+0000 mgr.smithi153.asomxr (mgr.14195) 153 : cluster 0 pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:13.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:13 smithi163 bash[22347]: cluster 2024-03-12T02:15:12.349517+0000 mgr.smithi153.asomxr (mgr.14195) 154 : cluster 0 pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:14.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:13 smithi153 bash[14340]: cluster 2024-03-12T02:15:12.349517+0000 mgr.smithi153.asomxr (mgr.14195) 154 : cluster 0 pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:14.876 INFO:teuthology.orchestra.run.smithi163.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi163/config 2024-03-12T02:15:15.920 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:15 smithi163 bash[22347]: cluster 2024-03-12T02:15:14.350179+0000 mgr.smithi153.asomxr (mgr.14195) 155 : cluster 0 pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:15.920 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:15 smithi163 bash[22347]: audit 2024-03-12T02:15:15.375662+0000 mon.smithi153 (mon.0) 704 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:15:16.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:15 smithi153 bash[14340]: cluster 2024-03-12T02:15:14.350179+0000 mgr.smithi153.asomxr (mgr.14195) 155 : cluster 0 pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:16.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:15 smithi153 bash[14340]: audit 2024-03-12T02:15:15.375662+0000 mon.smithi153 (mon.0) 704 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:15:17.981 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:17 smithi163 bash[22347]: cluster 2024-03-12T02:15:16.350785+0000 mgr.smithi153.asomxr (mgr.14195) 156 : cluster 0 pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:18.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:17 smithi153 bash[14340]: cluster 2024-03-12T02:15:16.350785+0000 mgr.smithi153.asomxr (mgr.14195) 156 : cluster 0 pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:18.377 INFO:teuthology.orchestra.run.smithi163.stdout:[client.1] 2024-03-12T02:15:18.378 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC2uu9laFgFFhAAr+lKZ2Uu97Pnxigwhtx/Yg== 2024-03-12T02:15:18.798 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:18 smithi163 bash[22347]: audit 2024-03-12T02:15:18.366682+0000 mon.smithi163 (mon.1) 18 : audit 1 from='client.? 172.21.15.163:0/4215439099' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T02:15:18.798 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:18 smithi163 bash[22347]: audit 2024-03-12T02:15:18.368974+0000 mon.smithi153 (mon.0) 705 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T02:15:18.798 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:18 smithi163 bash[22347]: audit 2024-03-12T02:15:18.374659+0000 mon.smithi153 (mon.0) 706 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-12T02:15:19.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:18 smithi153 bash[14340]: audit 2024-03-12T02:15:18.366682+0000 mon.smithi163 (mon.1) 18 : audit 1 from='client.? 172.21.15.163:0/4215439099' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T02:15:19.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:18 smithi153 bash[14340]: audit 2024-03-12T02:15:18.368974+0000 mon.smithi153 (mon.0) 705 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-12T02:15:19.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:18 smithi153 bash[14340]: audit 2024-03-12T02:15:18.374659+0000 mon.smithi153 (mon.0) 706 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-12T02:15:19.556 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T02:15:19.556 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-12T02:15:19.556 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-12T02:15:19.579 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-12T02:15:19.580 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-12T02:15:19.580 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph mgr dump --format=json 2024-03-12T02:15:19.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:19 smithi163 bash[22347]: cluster 2024-03-12T02:15:18.351268+0000 mgr.smithi153.asomxr (mgr.14195) 157 : cluster 0 pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:20.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:19 smithi153 bash[14340]: cluster 2024-03-12T02:15:18.351268+0000 mgr.smithi153.asomxr (mgr.14195) 157 : cluster 0 pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:22.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:21 smithi153 bash[14340]: cluster 2024-03-12T02:15:20.351792+0000 mgr.smithi153.asomxr (mgr.14195) 158 : cluster 0 pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:22.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:21 smithi153 bash[14340]: audit 2024-03-12T02:15:21.282562+0000 mon.smithi153 (mon.0) 707 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:15:22.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:21 smithi153 bash[14340]: audit 2024-03-12T02:15:21.290418+0000 mon.smithi153 (mon.0) 708 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:15:22.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:21 smithi163 bash[22347]: cluster 2024-03-12T02:15:20.351792+0000 mgr.smithi153.asomxr (mgr.14195) 158 : cluster 0 pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:21 smithi163 bash[22347]: audit 2024-03-12T02:15:21.282562+0000 mon.smithi153 (mon.0) 707 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:15:22.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:21 smithi163 bash[22347]: audit 2024-03-12T02:15:21.290418+0000 mon.smithi153 (mon.0) 708 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:15:23.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:23 smithi153 bash[14340]: audit 2024-03-12T02:15:22.260453+0000 mon.smithi153 (mon.0) 709 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:15:23.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:23 smithi153 bash[14340]: audit 2024-03-12T02:15:22.267471+0000 mon.smithi153 (mon.0) 710 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:15:23.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:23 smithi153 bash[14340]: cluster 2024-03-12T02:15:22.352430+0000 mgr.smithi153.asomxr (mgr.14195) 159 : cluster 0 pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:23.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:23 smithi163 bash[22347]: audit 2024-03-12T02:15:22.260453+0000 mon.smithi153 (mon.0) 709 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:15:23.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:23 smithi163 bash[22347]: audit 2024-03-12T02:15:22.267471+0000 mon.smithi153 (mon.0) 710 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:15:23.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:23 smithi163 bash[22347]: cluster 2024-03-12T02:15:22.352430+0000 mgr.smithi153.asomxr (mgr.14195) 159 : cluster 0 pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:24.717 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:25.682 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:25 smithi153 bash[14340]: cluster 2024-03-12T02:15:24.352998+0000 mgr.smithi153.asomxr (mgr.14195) 160 : cluster 0 pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:25.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:25 smithi163 bash[22347]: cluster 2024-03-12T02:15:24.352998+0000 mgr.smithi153.asomxr (mgr.14195) 160 : cluster 0 pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:27.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:27 smithi153 bash[14340]: cluster 2024-03-12T02:15:26.353527+0000 mgr.smithi153.asomxr (mgr.14195) 161 : cluster 0 pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:27.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:27 smithi163 bash[22347]: cluster 2024-03-12T02:15:26.353527+0000 mgr.smithi153.asomxr (mgr.14195) 161 : cluster 0 pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:27.973 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:15:28.773 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":19,"active_gid":14195,"active_name":"smithi153.asomxr","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":564758917},{"type":"v1","addr":"172.21.15.153:6801","nonce":564758917}]},"active_addr":"172.21.15.153:6801/564758917","active_change":"2024-03-12T02:12:00.299353+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14212,"name":"smithi163.fvmiae","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.153:8443/","prometheus":"http://172.21.15.153:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":545760385}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":3831019359}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2376951563}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":3528419328}]}]} 2024-03-12T02:15:28.777 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-12T02:15:28.777 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-12T02:15:28.777 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-12T02:15:28.962 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:28 smithi163 bash[22347]: audit 2024-03-12T02:15:27.959437+0000 mon.smithi153 (mon.0) 711 : audit 0 from='client.? 172.21.15.153:0/2921245997' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-12T02:15:29.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:28 smithi153 bash[14340]: audit 2024-03-12T02:15:27.959437+0000 mon.smithi153 (mon.0) 711 : audit 0 from='client.? 172.21.15.153:0/2921245997' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-12T02:15:29.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:29 smithi163 bash[22347]: cluster 2024-03-12T02:15:28.354085+0000 mgr.smithi153.asomxr (mgr.14195) 162 : cluster 0 pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:30.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:29 smithi153 bash[14340]: cluster 2024-03-12T02:15:28.354085+0000 mgr.smithi153.asomxr (mgr.14195) 162 : cluster 0 pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:30.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:30 smithi163 bash[22347]: audit 2024-03-12T02:15:30.375759+0000 mon.smithi153 (mon.0) 712 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:15:31.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:30 smithi153 bash[14340]: audit 2024-03-12T02:15:30.375759+0000 mon.smithi153 (mon.0) 712 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:15:31.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:31 smithi163 bash[22347]: cluster 2024-03-12T02:15:30.354647+0000 mgr.smithi153.asomxr (mgr.14195) 163 : cluster 0 pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:32.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:31 smithi153 bash[14340]: cluster 2024-03-12T02:15:30.354647+0000 mgr.smithi153.asomxr (mgr.14195) 163 : cluster 0 pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:33.523 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:33.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:33 smithi153 bash[14340]: cluster 2024-03-12T02:15:32.355213+0000 mgr.smithi153.asomxr (mgr.14195) 164 : cluster 0 pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:33.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:33 smithi163 bash[22347]: cluster 2024-03-12T02:15:32.355213+0000 mgr.smithi153.asomxr (mgr.14195) 164 : cluster 0 pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:35.821 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:35 smithi153 bash[14340]: cluster 2024-03-12T02:15:34.355829+0000 mgr.smithi153.asomxr (mgr.14195) 165 : cluster 0 pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:35.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:35 smithi163 bash[22347]: cluster 2024-03-12T02:15:34.355829+0000 mgr.smithi153.asomxr (mgr.14195) 165 : cluster 0 pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:36.672 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:15:36.672 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":43,"fsid":"5cbf0f4e-e015-11ee-95c8-87774f69a715","created":"2024-03-12T02:09:06.107943+0000","modified":"2024-03-12T02:14:37.939127+0000","last_up_change":"2024-03-12T02:14:33.911715+0000","last_in_change":"2024-03-12T02:13:39.749392+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":17,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T02:14:14.892829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad1d33d5-2e6b-4e4b-b148-3cb45de94e09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6801","nonce":3954737342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6803","nonce":3954737342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6807","nonce":3954737342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6805","nonce":3954737342}]},"public_addr":"172.21.15.163:6801/3954737342","cluster_addr":"172.21.15.163:6803/3954737342","heartbeat_back_addr":"172.21.15.163:6807/3954737342","heartbeat_front_addr":"172.21.15.163:6805/3954737342","state":["exists","up"]},{"osd":1,"uuid":"e789a733-1bcc-4a47-8c1f-6190d10a0dba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6803","nonce":799977908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6805","nonce":799977908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6809","nonce":799977908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6807","nonce":799977908}]},"public_addr":"172.21.15.153:6803/799977908","cluster_addr":"172.21.15.153:6805/799977908","heartbeat_back_addr":"172.21.15.153:6809/799977908","heartbeat_front_addr":"172.21.15.153:6807/799977908","state":["exists","up"]},{"osd":2,"uuid":"d17e4690-4fa9-45b1-8fe6-e62f6b2475ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6809","nonce":2994034967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6811","nonce":2994034967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6815","nonce":2994034967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6813","nonce":2994034967}]},"public_addr":"172.21.15.163:6809/2994034967","cluster_addr":"172.21.15.163:6811/2994034967","heartbeat_back_addr":"172.21.15.163:6815/2994034967","heartbeat_front_addr":"172.21.15.163:6813/2994034967","state":["exists","up"]},{"osd":3,"uuid":"fc7e829f-69ae-482f-9633-ba76231f39a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6811","nonce":347906783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6813","nonce":347906783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6817","nonce":347906783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6815","nonce":347906783}]},"public_addr":"172.21.15.153:6811/347906783","cluster_addr":"172.21.15.153:6813/347906783","heartbeat_back_addr":"172.21.15.153:6817/347906783","heartbeat_front_addr":"172.21.15.153:6815/347906783","state":["exists","up"]},{"osd":4,"uuid":"40f0bafb-ca9a-47ca-9ec2-1d03004e6efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6817","nonce":1978911710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6819","nonce":1978911710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6823","nonce":1978911710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6821","nonce":1978911710}]},"public_addr":"172.21.15.163:6817/1978911710","cluster_addr":"172.21.15.163:6819/1978911710","heartbeat_back_addr":"172.21.15.163:6823/1978911710","heartbeat_front_addr":"172.21.15.163:6821/1978911710","state":["exists","up"]},{"osd":5,"uuid":"301fd961-da29-4b85-8683-c117bb4a5086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6819","nonce":2985927744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6821","nonce":2985927744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6825","nonce":2985927744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6823","nonce":2985927744}]},"public_addr":"172.21.15.153:6819/2985927744","cluster_addr":"172.21.15.153:6821/2985927744","heartbeat_back_addr":"172.21.15.153:6825/2985927744","heartbeat_front_addr":"172.21.15.153:6823/2985927744","state":["exists","up"]},{"osd":6,"uuid":"e19436c1-f7ac-415b-b4be-1c2c52c87473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6825","nonce":1178116342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6827","nonce":1178116342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6831","nonce":1178116342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6829","nonce":1178116342}]},"public_addr":"172.21.15.163:6825/1178116342","cluster_addr":"172.21.15.163:6827/1178116342","heartbeat_back_addr":"172.21.15.163:6831/1178116342","heartbeat_front_addr":"172.21.15.163:6829/1178116342","state":["exists","up"]},{"osd":7,"uuid":"a76fe6b3-b2ae-4fab-b922-fd436208b6db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6827","nonce":3471588101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6829","nonce":3471588101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6833","nonce":3471588101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6831","nonce":3471588101}]},"public_addr":"172.21.15.153:6827/3471588101","cluster_addr":"172.21.15.153:6829/3471588101","heartbeat_back_addr":"172.21.15.153:6833/3471588101","heartbeat_front_addr":"172.21.15.153:6831/3471588101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:05.021168+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:10.860357+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:11.204898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:19.155151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:16.681764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:26.262881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:21.426567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:32.729454+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.153:6801/689479112":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/4010545582":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/3030920440":"2024-03-13T02:10:29.011431+0000","172.21.15.153:6801/1133623500":"2024-03-13T02:10:29.011431+0000","172.21.15.153:0/3201040853":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/178648720":"2024-03-13T02:10:29.011431+0000","172.21.15.153:0/1242805769":"2024-03-13T02:10:29.011431+0000","172.21.15.153:6800/689479112":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/1363796892":"2024-03-13T02:09:40.347470+0000","172.21.15.153:0/714144036":"2024-03-13T02:09:40.347470+0000","172.21.15.153:0/4185653963":"2024-03-13T02:09:40.347470+0000","172.21.15.153:0/2183141425":"2024-03-13T02:12:00.299133+0000","172.21.15.153:6800/296961988":"2024-03-13T02:09:40.347470+0000","172.21.15.153:6800/1133623500":"2024-03-13T02:10:29.011431+0000","172.21.15.153:6801/296961988":"2024-03-13T02:09:40.347470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T02:15:37.483 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-12T02:15:37.484 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-12T02:15:37.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:37 smithi153 bash[14340]: cluster 2024-03-12T02:15:36.356263+0000 mgr.smithi153.asomxr (mgr.14195) 166 : cluster 0 pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:37.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:37 smithi153 bash[14340]: audit 2024-03-12T02:15:36.667881+0000 mon.smithi153 (mon.0) 713 : audit 0 from='client.? 172.21.15.153:0/4133885106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T02:15:37.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:37 smithi163 bash[22347]: cluster 2024-03-12T02:15:36.356263+0000 mgr.smithi153.asomxr (mgr.14195) 166 : cluster 0 pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:37.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:37 smithi163 bash[22347]: audit 2024-03-12T02:15:36.667881+0000 mon.smithi153 (mon.0) 713 : audit 0 from='client.? 172.21.15.153:0/4133885106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T02:15:39.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:39 smithi163 bash[22347]: cluster 2024-03-12T02:15:38.356883+0000 mgr.smithi153.asomxr (mgr.14195) 167 : cluster 0 pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:40.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:39 smithi153 bash[14340]: cluster 2024-03-12T02:15:38.356883+0000 mgr.smithi153.asomxr (mgr.14195) 167 : cluster 0 pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:41.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:41 smithi163 bash[22347]: cluster 2024-03-12T02:15:40.357630+0000 mgr.smithi153.asomxr (mgr.14195) 168 : cluster 0 pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:42.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:41 smithi153 bash[14340]: cluster 2024-03-12T02:15:40.357630+0000 mgr.smithi153.asomxr (mgr.14195) 168 : cluster 0 pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:42.238 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:43.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:43 smithi163 bash[22347]: cluster 2024-03-12T02:15:42.358069+0000 mgr.smithi153.asomxr (mgr.14195) 169 : cluster 0 pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:44.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:43 smithi153 bash[14340]: cluster 2024-03-12T02:15:42.358069+0000 mgr.smithi153.asomxr (mgr.14195) 169 : cluster 0 pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:45.456 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:15:45.456 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":43,"fsid":"5cbf0f4e-e015-11ee-95c8-87774f69a715","created":"2024-03-12T02:09:06.107943+0000","modified":"2024-03-12T02:14:37.939127+0000","last_up_change":"2024-03-12T02:14:33.911715+0000","last_in_change":"2024-03-12T02:13:39.749392+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":17,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-12T02:14:14.892829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ad1d33d5-2e6b-4e4b-b148-3cb45de94e09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6801","nonce":3954737342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6803","nonce":3954737342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6807","nonce":3954737342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3954737342},{"type":"v1","addr":"172.21.15.163:6805","nonce":3954737342}]},"public_addr":"172.21.15.163:6801/3954737342","cluster_addr":"172.21.15.163:6803/3954737342","heartbeat_back_addr":"172.21.15.163:6807/3954737342","heartbeat_front_addr":"172.21.15.163:6805/3954737342","state":["exists","up"]},{"osd":1,"uuid":"e789a733-1bcc-4a47-8c1f-6190d10a0dba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6803","nonce":799977908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6805","nonce":799977908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6809","nonce":799977908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":799977908},{"type":"v1","addr":"172.21.15.153:6807","nonce":799977908}]},"public_addr":"172.21.15.153:6803/799977908","cluster_addr":"172.21.15.153:6805/799977908","heartbeat_back_addr":"172.21.15.153:6809/799977908","heartbeat_front_addr":"172.21.15.153:6807/799977908","state":["exists","up"]},{"osd":2,"uuid":"d17e4690-4fa9-45b1-8fe6-e62f6b2475ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6809","nonce":2994034967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6811","nonce":2994034967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6815","nonce":2994034967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":2994034967},{"type":"v1","addr":"172.21.15.163:6813","nonce":2994034967}]},"public_addr":"172.21.15.163:6809/2994034967","cluster_addr":"172.21.15.163:6811/2994034967","heartbeat_back_addr":"172.21.15.163:6815/2994034967","heartbeat_front_addr":"172.21.15.163:6813/2994034967","state":["exists","up"]},{"osd":3,"uuid":"fc7e829f-69ae-482f-9633-ba76231f39a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6811","nonce":347906783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6813","nonce":347906783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6817","nonce":347906783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":347906783},{"type":"v1","addr":"172.21.15.153:6815","nonce":347906783}]},"public_addr":"172.21.15.153:6811/347906783","cluster_addr":"172.21.15.153:6813/347906783","heartbeat_back_addr":"172.21.15.153:6817/347906783","heartbeat_front_addr":"172.21.15.153:6815/347906783","state":["exists","up"]},{"osd":4,"uuid":"40f0bafb-ca9a-47ca-9ec2-1d03004e6efc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6817","nonce":1978911710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6819","nonce":1978911710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6823","nonce":1978911710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":1978911710},{"type":"v1","addr":"172.21.15.163:6821","nonce":1978911710}]},"public_addr":"172.21.15.163:6817/1978911710","cluster_addr":"172.21.15.163:6819/1978911710","heartbeat_back_addr":"172.21.15.163:6823/1978911710","heartbeat_front_addr":"172.21.15.163:6821/1978911710","state":["exists","up"]},{"osd":5,"uuid":"301fd961-da29-4b85-8683-c117bb4a5086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6819","nonce":2985927744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6821","nonce":2985927744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6825","nonce":2985927744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2985927744},{"type":"v1","addr":"172.21.15.153:6823","nonce":2985927744}]},"public_addr":"172.21.15.153:6819/2985927744","cluster_addr":"172.21.15.153:6821/2985927744","heartbeat_back_addr":"172.21.15.153:6825/2985927744","heartbeat_front_addr":"172.21.15.153:6823/2985927744","state":["exists","up"]},{"osd":6,"uuid":"e19436c1-f7ac-415b-b4be-1c2c52c87473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6825","nonce":1178116342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6827","nonce":1178116342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6831","nonce":1178116342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":1178116342},{"type":"v1","addr":"172.21.15.163:6829","nonce":1178116342}]},"public_addr":"172.21.15.163:6825/1178116342","cluster_addr":"172.21.15.163:6827/1178116342","heartbeat_back_addr":"172.21.15.163:6831/1178116342","heartbeat_front_addr":"172.21.15.163:6829/1178116342","state":["exists","up"]},{"osd":7,"uuid":"a76fe6b3-b2ae-4fab-b922-fd436208b6db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6827","nonce":3471588101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6829","nonce":3471588101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6833","nonce":3471588101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":3471588101},{"type":"v1","addr":"172.21.15.153:6831","nonce":3471588101}]},"public_addr":"172.21.15.153:6827/3471588101","cluster_addr":"172.21.15.153:6829/3471588101","heartbeat_back_addr":"172.21.15.153:6833/3471588101","heartbeat_front_addr":"172.21.15.153:6831/3471588101","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:05.021168+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:10.860357+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:11.204898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:19.155151+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:16.681764+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:26.262881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:21.426567+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-12T02:14:32.729454+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.153:6801/689479112":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/4010545582":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/3030920440":"2024-03-13T02:10:29.011431+0000","172.21.15.153:6801/1133623500":"2024-03-13T02:10:29.011431+0000","172.21.15.153:0/3201040853":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/178648720":"2024-03-13T02:10:29.011431+0000","172.21.15.153:0/1242805769":"2024-03-13T02:10:29.011431+0000","172.21.15.153:6800/689479112":"2024-03-13T02:12:00.299133+0000","172.21.15.153:0/1363796892":"2024-03-13T02:09:40.347470+0000","172.21.15.153:0/714144036":"2024-03-13T02:09:40.347470+0000","172.21.15.153:0/4185653963":"2024-03-13T02:09:40.347470+0000","172.21.15.153:0/2183141425":"2024-03-13T02:12:00.299133+0000","172.21.15.153:6800/296961988":"2024-03-13T02:09:40.347470+0000","172.21.15.153:6800/1133623500":"2024-03-13T02:10:29.011431+0000","172.21.15.153:6801/296961988":"2024-03-13T02:09:40.347470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-12T02:15:45.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:45 smithi153 bash[14340]: cluster 2024-03-12T02:15:44.358628+0000 mgr.smithi153.asomxr (mgr.14195) 170 : cluster 0 pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:45.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:45 smithi153 bash[14340]: audit 2024-03-12T02:15:45.375461+0000 mon.smithi153 (mon.0) 714 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:15:45.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:45 smithi153 bash[14340]: audit 2024-03-12T02:15:45.451484+0000 mon.smithi153 (mon.0) 715 : audit 0 from='client.? 172.21.15.153:0/374384140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T02:15:45.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:45 smithi163 bash[22347]: cluster 2024-03-12T02:15:44.358628+0000 mgr.smithi153.asomxr (mgr.14195) 170 : cluster 0 pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:45.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:45 smithi163 bash[22347]: audit 2024-03-12T02:15:45.375461+0000 mon.smithi153 (mon.0) 714 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:15:45.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:45 smithi163 bash[22347]: audit 2024-03-12T02:15:45.451484+0000 mon.smithi153 (mon.0) 715 : audit 0 from='client.? 172.21.15.153:0/374384140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-12T02:15:46.278 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-12T02:15:46.278 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-12T02:15:46.279 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-12T02:15:46.279 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-12T02:15:46.280 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-12T02:15:46.280 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-12T02:15:46.280 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-12T02:15:46.281 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-12T02:15:47.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:47 smithi163 bash[22347]: cluster 2024-03-12T02:15:46.359084+0000 mgr.smithi153.asomxr (mgr.14195) 171 : cluster 0 pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:48.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:47 smithi153 bash[14340]: cluster 2024-03-12T02:15:46.359084+0000 mgr.smithi153.asomxr (mgr.14195) 171 : cluster 0 pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:49.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:49 smithi163 bash[22347]: cluster 2024-03-12T02:15:48.359642+0000 mgr.smithi153.asomxr (mgr.14195) 172 : cluster 0 pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:50.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:49 smithi153 bash[14340]: cluster 2024-03-12T02:15:48.359642+0000 mgr.smithi153.asomxr (mgr.14195) 172 : cluster 0 pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:51.100 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:51.100 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:51.101 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:51.101 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:51.101 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:51.107 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:51.107 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:51.108 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:15:51.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:51 smithi163 bash[22347]: cluster 2024-03-12T02:15:50.360214+0000 mgr.smithi153.asomxr (mgr.14195) 173 : cluster 0 pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:52.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:51 smithi153 bash[14340]: cluster 2024-03-12T02:15:50.360214+0000 mgr.smithi153.asomxr (mgr.14195) 173 : cluster 0 pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:53.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:53 smithi163 bash[22347]: cluster 2024-03-12T02:15:52.360766+0000 mgr.smithi153.asomxr (mgr.14195) 174 : cluster 0 pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:54.047 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:53 smithi153 bash[14340]: cluster 2024-03-12T02:15:52.360766+0000 mgr.smithi153.asomxr (mgr.14195) 174 : cluster 0 pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:55.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:55 smithi153 bash[14340]: cluster 2024-03-12T02:15:54.361225+0000 mgr.smithi153.asomxr (mgr.14195) 175 : cluster 0 pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:55.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:55 smithi163 bash[22347]: cluster 2024-03-12T02:15:54.361225+0000 mgr.smithi153.asomxr (mgr.14195) 175 : cluster 0 pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:57.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:57 smithi163 bash[22347]: cluster 2024-03-12T02:15:56.361638+0000 mgr.smithi153.asomxr (mgr.14195) 176 : cluster 0 pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:58.038 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:57 smithi153 bash[14340]: cluster 2024-03-12T02:15:56.361638+0000 mgr.smithi153.asomxr (mgr.14195) 176 : cluster 0 pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:15:59.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:15:59 smithi163 bash[22347]: cluster 2024-03-12T02:15:58.362123+0000 mgr.smithi153.asomxr (mgr.14195) 177 : cluster 0 pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:00.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:15:59 smithi153 bash[14340]: cluster 2024-03-12T02:15:58.362123+0000 mgr.smithi153.asomxr (mgr.14195) 177 : cluster 0 pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:00.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:00 smithi163 bash[22347]: audit 2024-03-12T02:16:00.376246+0000 mon.smithi153 (mon.0) 716 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:16:01.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:00 smithi153 bash[14340]: audit 2024-03-12T02:16:00.376246+0000 mon.smithi153 (mon.0) 716 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:16:01.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:01 smithi163 bash[22347]: cluster 2024-03-12T02:16:00.362536+0000 mgr.smithi153.asomxr (mgr.14195) 178 : cluster 0 pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:02.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:01 smithi153 bash[14340]: cluster 2024-03-12T02:16:00.362536+0000 mgr.smithi153.asomxr (mgr.14195) 178 : cluster 0 pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:03.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:03 smithi163 bash[22347]: cluster 2024-03-12T02:16:02.363015+0000 mgr.smithi153.asomxr (mgr.14195) 179 : cluster 0 pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:04.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:03 smithi153 bash[14340]: cluster 2024-03-12T02:16:02.363015+0000 mgr.smithi153.asomxr (mgr.14195) 179 : cluster 0 pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:04.644 INFO:teuthology.orchestra.run.smithi153.stdout:133143986199 2024-03-12T02:16:04.644 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-12T02:16:05.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:05 smithi163 bash[22347]: cluster 2024-03-12T02:16:04.363564+0000 mgr.smithi153.asomxr (mgr.14195) 180 : cluster 0 pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:06.029 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:05 smithi153 bash[14340]: cluster 2024-03-12T02:16:04.363564+0000 mgr.smithi153.asomxr (mgr.14195) 180 : cluster 0 pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:06.873 INFO:teuthology.orchestra.run.smithi153.stdout:167503724565 2024-03-12T02:16:06.874 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-12T02:16:06.900 INFO:teuthology.orchestra.run.smithi153.stdout:124554051608 2024-03-12T02:16:06.901 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-12T02:16:06.901 INFO:teuthology.orchestra.run.smithi153.stdout:85899345945 2024-03-12T02:16:06.902 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-12T02:16:07.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:07 smithi153 bash[14340]: cluster 2024-03-12T02:16:06.364111+0000 mgr.smithi153.asomxr (mgr.14195) 181 : cluster 0 pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:07.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:07 smithi163 bash[22347]: cluster 2024-03-12T02:16:06.364111+0000 mgr.smithi153.asomxr (mgr.14195) 181 : cluster 0 pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:08.490 INFO:teuthology.orchestra.run.smithi153.stdout:150323855383 2024-03-12T02:16:08.490 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-12T02:16:09.134 INFO:teuthology.orchestra.run.smithi153.stdout:90194313242 2024-03-12T02:16:09.134 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-12T02:16:09.171 INFO:teuthology.orchestra.run.smithi153.stdout:68719476762 2024-03-12T02:16:09.171 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-12T02:16:09.366 INFO:teuthology.orchestra.run.smithi153.stdout:111669149720 2024-03-12T02:16:09.367 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-12T02:16:09.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:09 smithi163 bash[22347]: cluster 2024-03-12T02:16:08.364658+0000 mgr.smithi153.asomxr (mgr.14195) 182 : cluster 0 pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:10.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:09 smithi153 bash[14340]: cluster 2024-03-12T02:16:08.364658+0000 mgr.smithi153.asomxr (mgr.14195) 182 : cluster 0 pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:11.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:11 smithi163 bash[22347]: cluster 2024-03-12T02:16:10.365272+0000 mgr.smithi153.asomxr (mgr.14195) 183 : cluster 0 pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:12.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:11 smithi153 bash[14340]: cluster 2024-03-12T02:16:10.365272+0000 mgr.smithi153.asomxr (mgr.14195) 183 : cluster 0 pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:12.714 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:16:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:16:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:16:13.740 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:16:13.741 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:16:13.742 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:16:13.743 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:16:13.743 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:16:13.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:13 smithi163 bash[22347]: cluster 2024-03-12T02:16:12.365783+0000 mgr.smithi153.asomxr (mgr.14195) 184 : cluster 0 pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:14.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:13 smithi153 bash[14340]: cluster 2024-03-12T02:16:12.365783+0000 mgr.smithi153.asomxr (mgr.14195) 184 : cluster 0 pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:15 smithi163 bash[22347]: cluster 2024-03-12T02:16:14.366277+0000 mgr.smithi153.asomxr (mgr.14195) 185 : cluster 0 pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:15.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:15 smithi163 bash[22347]: audit 2024-03-12T02:16:15.376285+0000 mon.smithi153 (mon.0) 717 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:16:16.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:15 smithi153 bash[14340]: cluster 2024-03-12T02:16:14.366277+0000 mgr.smithi153.asomxr (mgr.14195) 185 : cluster 0 pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:16.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:15 smithi153 bash[14340]: audit 2024-03-12T02:16:15.376285+0000 mon.smithi153 (mon.0) 717 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:16:17.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:17 smithi163 bash[22347]: cluster 2024-03-12T02:16:16.366787+0000 mgr.smithi153.asomxr (mgr.14195) 186 : cluster 0 pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:18.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:17 smithi153 bash[14340]: cluster 2024-03-12T02:16:16.366787+0000 mgr.smithi153.asomxr (mgr.14195) 186 : cluster 0 pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:19.927 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:19 smithi153 bash[14340]: cluster 2024-03-12T02:16:18.367138+0000 mgr.smithi153.asomxr (mgr.14195) 187 : cluster 0 pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:19.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:19 smithi163 bash[22347]: cluster 2024-03-12T02:16:18.367138+0000 mgr.smithi153.asomxr (mgr.14195) 187 : cluster 0 pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:21.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:21 smithi163 bash[22347]: cluster 2024-03-12T02:16:20.367613+0000 mgr.smithi153.asomxr (mgr.14195) 188 : cluster 0 pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:21.995 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:21 smithi153 bash[14340]: cluster 2024-03-12T02:16:20.367613+0000 mgr.smithi153.asomxr (mgr.14195) 188 : cluster 0 pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:22.229 INFO:teuthology.orchestra.run.smithi153.stdout:133143986203 2024-03-12T02:16:22.958 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:22 smithi153 bash[14340]: audit 2024-03-12T02:16:22.226242+0000 mon.smithi153 (mon.0) 718 : audit 0 from='client.? 172.21.15.153:0/1733160448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-12T02:16:22.958 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:22 smithi153 bash[14340]: audit 2024-03-12T02:16:22.367907+0000 mon.smithi153 (mon.0) 719 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:16:23.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:22 smithi163 bash[22347]: audit 2024-03-12T02:16:22.226242+0000 mon.smithi153 (mon.0) 718 : audit 0 from='client.? 172.21.15.153:0/1733160448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-12T02:16:23.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:22 smithi163 bash[22347]: audit 2024-03-12T02:16:22.367907+0000 mon.smithi153 (mon.0) 719 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:16:24.050 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:23 smithi153 bash[14340]: cluster 2024-03-12T02:16:22.367928+0000 mgr.smithi153.asomxr (mgr.14195) 189 : cluster 0 pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:24.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:23 smithi163 bash[22347]: cluster 2024-03-12T02:16:22.367928+0000 mgr.smithi153.asomxr (mgr.14195) 189 : cluster 0 pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:26.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:25 smithi153 bash[14340]: cluster 2024-03-12T02:16:24.368298+0000 mgr.smithi153.asomxr (mgr.14195) 190 : cluster 0 pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:26.232 INFO:teuthology.orchestra.run.smithi153.stdout:167503724569 2024-03-12T02:16:26.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:25 smithi163 bash[22347]: cluster 2024-03-12T02:16:24.368298+0000 mgr.smithi153.asomxr (mgr.14195) 190 : cluster 0 pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:26.767 INFO:teuthology.orchestra.run.smithi153.stdout:85899345949 2024-03-12T02:16:27.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:26 smithi153 bash[14340]: audit 2024-03-12T02:16:26.229669+0000 mon.smithi153 (mon.0) 720 : audit 0 from='client.? 172.21.15.153:0/2845749212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-12T02:16:27.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:26 smithi163 bash[22347]: audit 2024-03-12T02:16:26.229669+0000 mon.smithi153 (mon.0) 720 : audit 0 from='client.? 172.21.15.153:0/2845749212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-12T02:16:27.315 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986199 got 133143986203 for osd.6 2024-03-12T02:16:27.315 DEBUG:teuthology.parallel:result is None 2024-03-12T02:16:28.013 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:27 smithi153 bash[14340]: cluster 2024-03-12T02:16:26.368757+0000 mgr.smithi153.asomxr (mgr.14195) 191 : cluster 0 pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:28.013 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:27 smithi153 bash[14340]: audit 2024-03-12T02:16:26.764067+0000 mon.smithi153 (mon.0) 721 : audit 0 from='client.? 172.21.15.153:0/1317982761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-12T02:16:28.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:27 smithi163 bash[22347]: cluster 2024-03-12T02:16:26.368757+0000 mgr.smithi153.asomxr (mgr.14195) 191 : cluster 0 pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:28.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:27 smithi163 bash[22347]: audit 2024-03-12T02:16:26.764067+0000 mon.smithi153 (mon.0) 721 : audit 0 from='client.? 172.21.15.153:0/1317982761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-12T02:16:28.357 INFO:teuthology.orchestra.run.smithi153.stdout:90194313246 2024-03-12T02:16:29.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:29 smithi153 bash[14340]: audit 2024-03-12T02:16:28.009552+0000 mon.smithi153 (mon.0) 722 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:16:29.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:29 smithi153 bash[14340]: audit 2024-03-12T02:16:28.016844+0000 mon.smithi153 (mon.0) 723 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:16:29.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:29 smithi153 bash[14340]: audit 2024-03-12T02:16:28.353963+0000 mon.smithi153 (mon.0) 724 : audit 0 from='client.? 172.21.15.153:0/3999426599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-12T02:16:29.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:29 smithi153 bash[14340]: cluster 2024-03-12T02:16:28.369325+0000 mgr.smithi153.asomxr (mgr.14195) 192 : cluster 0 pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:29.389 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724565 got 167503724569 for osd.7 2024-03-12T02:16:29.389 DEBUG:teuthology.parallel:result is None 2024-03-12T02:16:29.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:29 smithi163 bash[22347]: audit 2024-03-12T02:16:28.009552+0000 mon.smithi153 (mon.0) 722 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:16:29.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:29 smithi163 bash[22347]: audit 2024-03-12T02:16:28.016844+0000 mon.smithi153 (mon.0) 723 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:16:29.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:29 smithi163 bash[22347]: audit 2024-03-12T02:16:28.353963+0000 mon.smithi153 (mon.0) 724 : audit 0 from='client.? 172.21.15.153:0/3999426599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-12T02:16:29.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:29 smithi163 bash[22347]: cluster 2024-03-12T02:16:28.369325+0000 mgr.smithi153.asomxr (mgr.14195) 192 : cluster 0 pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:29.989 INFO:teuthology.orchestra.run.smithi153.stdout:150323855387 2024-03-12T02:16:30.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:30 smithi153 bash[14340]: audit 2024-03-12T02:16:29.986848+0000 mon.smithi153 (mon.0) 725 : audit 0 from='client.? 172.21.15.153:0/2933524599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-12T02:16:30.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:30 smithi163 bash[22347]: audit 2024-03-12T02:16:29.986848+0000 mon.smithi153 (mon.0) 725 : audit 0 from='client.? 172.21.15.153:0/2933524599' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-12T02:16:30.563 INFO:tasks.cephadm.ceph_manager.ceph:need seq 85899345945 got 85899345949 for osd.1 2024-03-12T02:16:30.563 DEBUG:teuthology.parallel:result is None 2024-03-12T02:16:30.997 INFO:teuthology.orchestra.run.smithi153.stdout:111669149725 2024-03-12T02:16:31.136 INFO:teuthology.orchestra.run.smithi153.stdout:68719476767 2024-03-12T02:16:31.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:31 smithi153 bash[14340]: cluster 2024-03-12T02:16:30.369786+0000 mgr.smithi153.asomxr (mgr.14195) 193 : cluster 0 pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:31.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:31 smithi153 bash[14340]: audit 2024-03-12T02:16:30.376422+0000 mon.smithi153 (mon.0) 726 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:16:31.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:31 smithi153 bash[14340]: audit 2024-03-12T02:16:30.994090+0000 mon.smithi153 (mon.0) 727 : audit 0 from='client.? 172.21.15.153:0/1840672211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-12T02:16:31.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:31 smithi163 bash[22347]: cluster 2024-03-12T02:16:30.369786+0000 mgr.smithi153.asomxr (mgr.14195) 193 : cluster 0 pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:31.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:31 smithi163 bash[22347]: audit 2024-03-12T02:16:30.376422+0000 mon.smithi153 (mon.0) 726 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:16:31.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:31 smithi163 bash[22347]: audit 2024-03-12T02:16:30.994090+0000 mon.smithi153 (mon.0) 727 : audit 0 from='client.? 172.21.15.153:0/1840672211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-12T02:16:32.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:32 smithi153 bash[14340]: audit 2024-03-12T02:16:31.133288+0000 mon.smithi153 (mon.0) 728 : audit 0 from='client.? 172.21.15.153:0/2555116344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-12T02:16:32.331 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855383 got 150323855387 for osd.5 2024-03-12T02:16:32.331 DEBUG:teuthology.parallel:result is None 2024-03-12T02:16:32.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:32 smithi163 bash[22347]: audit 2024-03-12T02:16:31.133288+0000 mon.smithi153 (mon.0) 728 : audit 0 from='client.? 172.21.15.153:0/2555116344' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-12T02:16:32.516 INFO:tasks.cephadm.ceph_manager.ceph:need seq 90194313242 got 90194313246 for osd.2 2024-03-12T02:16:32.516 DEBUG:teuthology.parallel:result is None 2024-03-12T02:16:33.313 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476762 got 68719476767 for osd.0 2024-03-12T02:16:33.313 DEBUG:teuthology.parallel:result is None 2024-03-12T02:16:33.369 INFO:teuthology.orchestra.run.smithi153.stdout:124554051614 2024-03-12T02:16:33.594 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149720 got 111669149725 for osd.4 2024-03-12T02:16:33.595 DEBUG:teuthology.parallel:result is None 2024-03-12T02:16:33.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:33 smithi163 bash[22347]: cluster 2024-03-12T02:16:32.370265+0000 mgr.smithi153.asomxr (mgr.14195) 194 : cluster 0 pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:33.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:33 smithi163 bash[22347]: audit 2024-03-12T02:16:33.366058+0000 mon.smithi153 (mon.0) 729 : audit 0 from='client.? 172.21.15.153:0/557106289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-12T02:16:34.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:33 smithi153 bash[14340]: cluster 2024-03-12T02:16:32.370265+0000 mgr.smithi153.asomxr (mgr.14195) 194 : cluster 0 pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:34.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:33 smithi153 bash[14340]: audit 2024-03-12T02:16:33.366058+0000 mon.smithi153 (mon.0) 729 : audit 0 from='client.? 172.21.15.153:0/557106289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-12T02:16:34.214 INFO:tasks.cephadm.ceph_manager.ceph:need seq 124554051608 got 124554051614 for osd.3 2024-03-12T02:16:34.214 DEBUG:teuthology.parallel:result is None 2024-03-12T02:16:34.215 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-12T02:16:34.215 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-12T02:16:35.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:35 smithi163 bash[22347]: cluster 2024-03-12T02:16:34.370727+0000 mgr.smithi153.asomxr (mgr.14195) 195 : cluster 0 pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:36.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:35 smithi153 bash[14340]: cluster 2024-03-12T02:16:34.370727+0000 mgr.smithi153.asomxr (mgr.14195) 195 : cluster 0 pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:37.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:37 smithi163 bash[22347]: cluster 2024-03-12T02:16:36.371299+0000 mgr.smithi153.asomxr (mgr.14195) 196 : cluster 0 pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:38.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:37 smithi153 bash[14340]: cluster 2024-03-12T02:16:36.371299+0000 mgr.smithi153.asomxr (mgr.14195) 196 : cluster 0 pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:38.150 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:16:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:39 smithi153 bash[14340]: cluster 2024-03-12T02:16:38.371645+0000 mgr.smithi153.asomxr (mgr.14195) 197 : cluster 0 pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:39 smithi153 bash[14340]: audit 2024-03-12T02:16:38.495871+0000 mon.smithi153 (mon.0) 730 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:16:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:39 smithi153 bash[14340]: audit 2024-03-12T02:16:38.504181+0000 mon.smithi153 (mon.0) 731 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:16:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:39 smithi153 bash[14340]: audit 2024-03-12T02:16:38.975730+0000 mon.smithi153 (mon.0) 732 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:16:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:39 smithi153 bash[14340]: audit 2024-03-12T02:16:38.976972+0000 mon.smithi153 (mon.0) 733 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:16:39.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:39 smithi153 bash[14340]: audit 2024-03-12T02:16:38.983360+0000 mon.smithi153 (mon.0) 734 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:16:39.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:39 smithi153 bash[14340]: audit 2024-03-12T02:16:38.989128+0000 mon.smithi153 (mon.0) 735 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:16:39.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:39 smithi163 bash[22347]: cluster 2024-03-12T02:16:38.371645+0000 mgr.smithi153.asomxr (mgr.14195) 197 : cluster 0 pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:39 smithi163 bash[22347]: audit 2024-03-12T02:16:38.495871+0000 mon.smithi153 (mon.0) 730 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:16:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:39 smithi163 bash[22347]: audit 2024-03-12T02:16:38.504181+0000 mon.smithi153 (mon.0) 731 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:16:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:39 smithi163 bash[22347]: audit 2024-03-12T02:16:38.975730+0000 mon.smithi153 (mon.0) 732 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:16:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:39 smithi163 bash[22347]: audit 2024-03-12T02:16:38.976972+0000 mon.smithi153 (mon.0) 733 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:16:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:39 smithi163 bash[22347]: audit 2024-03-12T02:16:38.983360+0000 mon.smithi153 (mon.0) 734 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:16:39.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:39 smithi163 bash[22347]: audit 2024-03-12T02:16:38.989128+0000 mon.smithi153 (mon.0) 735 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:16:41.441 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:16:41.443 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2024-03-12T02:16:41.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:41 smithi153 bash[14340]: cluster 2024-03-12T02:16:40.372301+0000 mgr.smithi153.asomxr (mgr.14195) 198 : cluster 0 pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:41.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:41 smithi163 bash[22347]: cluster 2024-03-12T02:16:40.372301+0000 mgr.smithi153.asomxr (mgr.14195) 198 : cluster 0 pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:42.267 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-03-12T02:16:40.371899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":160,"num_read_kb":139,"num_write":269,"num_write_kb":5026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":749928448,"kb_used":221312,"kb_used_data":5128,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":749707136,"statfs":{"total":767926730752,"available":767700107264,"internally_reserved":0,"allocated":5251072,"data_stored":3181887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002894"},"pg_stats":[{"pgid":"1.0","version":"33'217","reported_seq":331,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-12T02:14:39.916835+0000","last_change":"2024-03-12T02:14:37.950323+0000","last_active":"2024-03-12T02:14:39.916835+0000","last_peered":"2024-03-12T02:14:39.916835+0000","last_clean":"2024-03-12T02:14:39.916835+0000","last_became_active":"2024-03-12T02:14:37.949936+0000","last_became_peered":"2024-03-12T02:14:37.949936+0000","last_unstale":"2024-03-12T02:14:39.916835+0000","last_undegraded":"2024-03-12T02:14:39.916835+0000","last_fullsized":"2024-03-12T02:14:39.916835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T02:14:14.924429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T02:14:14.924429+0000","last_clean_scrub_stamp":"2024-03-12T02:14:14.924429+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:47:15.243158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":160,"num_read_kb":139,"num_write":269,"num_write_kb":5026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":160,"num_read_kb":139,"num_write":269,"num_write_kb":5026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":3,"acting":3,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":39,"seq":167503724573,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713032,"statfs":{"total":95990841344,"available":95962144768,"internally_reserved":0,"allocated":1028096,"data_stored":766750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85799999999999998,"15min":0.871},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1579999999999999,"5min":1.351,"15min":1.351},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.84399999999999997,"15min":0.84899999999999998},"min":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.006,"5min":1.21,"15min":1.21},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.35499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2649999999999999,"5min":1.365,"15min":1.365},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.97699999999999998}]},{"osd":2,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.88800000000000001,"15min":0.89400000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.97399999999999998,"15min":0.98799999999999999},"min":{"1min":0.621,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.2909999999999999,"5min":1.381,"15min":1.381},"last":0.79900000000000004}]},{"osd":3,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.876,"15min":0.89100000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2629999999999999,"5min":1.659,"15min":1.659},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.90400000000000003,"15min":0.91700000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1639999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.73599999999999999}]},{"osd":4,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.97399999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1879999999999999,"5min":1.272,"15min":1.272},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0329999999999999,"15min":1.0489999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2270000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.91600000000000004,"15min":0.92800000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.96999999999999997,"15min":0.98399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.81999999999999995}]},{"osd":6,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0669999999999999,"15min":1.085},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2450000000000001,"5min":1.488,"15min":1.488},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1020000000000001,"15min":1.1180000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3660000000000001,"5min":1.671,"15min":1.671},"last":0.86199999999999999}]}]},{"osd":5,"up_from":35,"seq":150323855390,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713612,"statfs":{"total":95990841344,"available":95962738688,"internally_reserved":0,"allocated":434176,"data_stored":176382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.85999999999999999,"15min":0.87},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1,"5min":1.5,"15min":1.5},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.92100000000000004,"15min":0.93799999999999994},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2669999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.78400000000000003}]},{"osd":1,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.72899999999999998,"15min":0.74199999999999999},"min":{"1min":0.434,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.48899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.115,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.71199999999999997}]},{"osd":2,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.86199999999999999,"15min":0.86599999999999999},"min":{"1min":0.623,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.89100000000000001,"15min":0.89200000000000002},"min":{"1min":0.63400000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.93700000000000006}]},{"osd":3,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.82199999999999995,"15min":0.83099999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1419999999999999,"5min":1.274,"15min":1.274},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.755,"15min":0.749},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.238,"5min":1.268,"15min":1.268},"last":0.621}]},{"osd":4,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94599999999999995,"15min":0.94399999999999995},"min":{"1min":0.68500000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1699999999999999,"5min":1.325,"15min":1.325},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.016,"15min":1.024},"min":{"1min":0.78900000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.224,"5min":1.714,"15min":1.714},"last":1.113}]},{"osd":6,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.0029999999999999,"15min":1.0069999999999999},"min":{"1min":0.77100000000000002,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.1619999999999999,"5min":1.494,"15min":1.494},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.97699999999999998,"15min":0.97599999999999998},"min":{"1min":0.73099999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.21,"5min":1.413,"15min":1.413},"last":0.97899999999999998}]},{"osd":7,"last update":"Tue Mar 12 02:15:38 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.244}]}]},{"osd":1,"up_from":20,"seq":85899345952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27444,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713612,"statfs":{"total":95990841344,"available":95962738688,"internally_reserved":0,"allocated":430080,"data_stored":176109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.91900000000000004,"15min":0.92900000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.127,"5min":1.321,"15min":1.321},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.89600000000000002,"15min":0.89100000000000001},"min":{"1min":0.59799999999999998,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.1859999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.1080000000000001}]},{"osd":2,"last update":"Tue Mar 12 02:16:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84899999999999998,"15min":0.84699999999999998},"min":{"1min":0.51500000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0900000000000001,"5min":1.284,"15min":1.284},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.91200000000000003,"15min":0.91000000000000003},"min":{"1min":0.64800000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1080000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.96099999999999997}]},{"osd":3,"last update":"Tue Mar 12 02:16:23 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.77200000000000002,"15min":0.77300000000000002},"min":{"1min":0.46200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":1.01},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.434,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97099999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.91700000000000004}]},{"osd":4,"last update":"Tue Mar 12 02:16:23 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.92100000000000004,"15min":0.92700000000000005},"min":{"1min":0.66800000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.069,"5min":1.294,"15min":1.294},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1,"15min":1.01},"min":{"1min":0.65200000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.2370000000000001,"5min":1.359,"15min":1.359},"last":1.131}]},{"osd":5,"last update":"Tue Mar 12 02:16:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.91300000000000003,"15min":0.93300000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3129999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.23},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.97499999999999998,"15min":0.996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1819999999999999,"5min":1.385,"15min":1.385},"last":0.86899999999999999}]},{"osd":6,"last update":"Tue Mar 12 02:16:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0960000000000001,"15min":1.1100000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4319999999999999,"5min":1.458,"15min":1.458},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.0860000000000001,"15min":1.093},"min":{"1min":0.69399999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.3029999999999999,"5min":1.484,"15min":1.484},"last":1.159}]},{"osd":7,"last update":"Tue Mar 12 02:15:37 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.204},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.1779999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713024,"statfs":{"total":95990841344,"available":95962136576,"internally_reserved":0,"allocated":1028096,"data_stored":766750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Mar 12 02:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81899999999999995,"15min":0.81699999999999995},"min":{"1min":0.58199999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.82299999999999995,"15min":0.81499999999999995},"min":{"1min":0.52500000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":1.0649999999999999}]},{"osd":2,"last update":"Tue Mar 12 02:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.70199999999999996,"15min":0.68700000000000006},"min":{"1min":0.54100000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":1.002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.64600000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1479999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.74099999999999999}]},{"osd":3,"last update":"Tue Mar 12 02:16:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91900000000000004,"15min":0.92700000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.1879999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.129}]},{"osd":4,"last update":"Tue Mar 12 02:16:24 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94499999999999995,"15min":0.95299999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2310000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.88600000000000001,"15min":0.89200000000000002},"min":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.151,"5min":1.244,"15min":1.244},"last":0.83999999999999997}]},{"osd":5,"last update":"Tue Mar 12 02:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0860000000000001,"15min":1.083},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.96399999999999997,"15min":0.96899999999999997},"min":{"1min":0.70299999999999996,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.1910000000000001,"5min":1.339,"15min":1.339},"last":0.97499999999999998}]},{"osd":6,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.98299999999999998,"15min":0.98699999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.161,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.036},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.087},"min":{"1min":0.69699999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.1970000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.288}]},{"osd":7,"last update":"Tue Mar 12 02:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.107,"15min":1.1100000000000001},"min":{"1min":0.80200000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.3899999999999999,"5min":1.613,"15min":1.613},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.163,"15min":1.1619999999999999},"min":{"1min":0.92100000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.2130000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713604,"statfs":{"total":95990841344,"available":95962730496,"internally_reserved":0,"allocated":434176,"data_stored":176382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.628},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78100000000000003,"15min":0.77500000000000002},"min":{"1min":0.53700000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.92000000000000004}]},{"osd":1,"last update":"Tue Mar 12 02:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.85799999999999998,"15min":0.86599999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.002,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.91300000000000003,"15min":0.91900000000000004},"min":{"1min":0.59699999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1699999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.84199999999999997}]},{"osd":3,"last update":"Tue Mar 12 02:16:23 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89700000000000002,"15min":0.89900000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.123,"5min":1.258,"15min":1.258},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.92100000000000004,"15min":0.92700000000000005},"min":{"1min":0.58599999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1439999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.1100000000000001}]},{"osd":4,"last update":"Tue Mar 12 02:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.92400000000000004,"15min":0.92700000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.87,"5min":0.92700000000000005,"15min":0.93700000000000006},"min":{"1min":0.59699999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1439999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.2290000000000001}]},{"osd":5,"last update":"Tue Mar 12 02:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.038,"15min":1.0489999999999999},"min":{"1min":0.69899999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.206},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.014,"15min":1.0249999999999999},"min":{"1min":0.75800000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.2689999999999999,"5min":1.383,"15min":1.383},"last":1.153}]},{"osd":6,"last update":"Tue Mar 12 02:16:23 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.98799999999999999,"15min":0.97999999999999998},"min":{"1min":0.83399999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2450000000000001,"5min":1.339,"15min":1.339},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.0580000000000001,"15min":1.0509999999999999},"min":{"1min":0.86399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.1850000000000001}]},{"osd":7,"last update":"Tue Mar 12 02:15:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.28}]}]},{"osd":4,"up_from":26,"seq":111669149727,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713612,"statfs":{"total":95990841344,"available":95962738688,"internally_reserved":0,"allocated":434176,"data_stored":176382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:20 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.54600000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.622},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.66400000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80900000000000005}]},{"osd":1,"last update":"Tue Mar 12 02:16:20 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.879,"15min":0.873},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3029999999999999,"5min":1.391,"15min":1.391},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.0569999999999999}]},{"osd":2,"last update":"Tue Mar 12 02:16:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77600000000000002,"15min":0.77200000000000002},"min":{"1min":0.60699999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.875,"15min":0.86399999999999999},"min":{"1min":0.71599999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2270000000000001,"5min":1.306,"15min":1.306},"last":0.71199999999999997}]},{"osd":3,"last update":"Tue Mar 12 02:16:25 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.96399999999999997,"15min":0.95599999999999996},"min":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.1819999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.98999999999999999,"15min":0.99399999999999999},"min":{"1min":0.75700000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.2350000000000001,"5min":1.353,"15min":1.353},"last":0.92100000000000004}]},{"osd":5,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0469999999999999,"15min":1.044},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.052,"5min":1.0740000000000001,"15min":1.0780000000000001},"min":{"1min":0.84499999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.3480000000000001,"5min":1.47,"15min":1.47},"last":0.77700000000000002}]},{"osd":6,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0880000000000001,"15min":1.0900000000000001},"min":{"1min":0.83199999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.381,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0649999999999999,"15min":1.0580000000000001},"min":{"1min":0.79800000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.502,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.85899999999999999}]},{"osd":7,"last update":"Tue Mar 12 02:16:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1619999999999999,"15min":1.159},"min":{"1min":0.878,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.131},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.149,"15min":1.151},"min":{"1min":0.95199999999999996,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.98899999999999999}]}]},{"osd":3,"up_from":29,"seq":124554051615,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713028,"statfs":{"total":95990841344,"available":95962140672,"internally_reserved":0,"allocated":1028096,"data_stored":766750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.82399999999999995,"15min":0.83399999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98399999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.55600000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.129,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.024}]},{"osd":1,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68999999999999995,"15min":0.69899999999999995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.02,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.79400000000000004,"15min":0.80600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.018,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86799999999999999,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.92900000000000005,"15min":0.93100000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.202,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.997}]},{"osd":4,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.3,"5min":1.387,"15min":1.387},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0269999999999999,"15min":1.034},"min":{"1min":0.63700000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1579999999999999,"5min":1.587,"15min":1.587},"last":0.96799999999999997}]},{"osd":5,"last update":"Tue Mar 12 02:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.93300000000000005,"15min":0.95399999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.3180000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.97399999999999998,"15min":1.002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3660000000000001,"5min":1.468,"15min":1.468},"last":1.2829999999999999}]},{"osd":6,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.117,"15min":1.1379999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.2609999999999999,"5min":1.52,"15min":1.52},"last":1.208},{"interface":"front","average":{"1min":1.034,"5min":1.119,"15min":1.133},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.5069999999999999,"5min":1.532,"15min":1.532},"last":1.1559999999999999}]},{"osd":7,"last update":"Tue Mar 12 02:15:39 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.2410000000000001}]}]},{"osd":6,"up_from":31,"seq":133143986207,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713612,"statfs":{"total":95990841344,"available":95962738688,"internally_reserved":0,"allocated":434176,"data_stored":176382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81899999999999995,"15min":0.81799999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.325,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.74099999999999999}]},{"osd":1,"last update":"Tue Mar 12 02:16:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86499999999999999,"15min":0.872},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.03,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89300000000000002,"15min":0.89400000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1080000000000001,"5min":1.131,"15min":1.131},"last":0.621}]},{"osd":2,"last update":"Tue Mar 12 02:16:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85699999999999998,"15min":0.84599999999999997},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.91600000000000004,"15min":0.92400000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2270000000000001,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.0149999999999999}]},{"osd":3,"last update":"Tue Mar 12 02:16:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.98399999999999999,"15min":0.996},"min":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.1020000000000001,"5min":1.627,"15min":1.627},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.999,"15min":1.0109999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.1899999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.88800000000000001}]},{"osd":4,"last update":"Tue Mar 12 02:16:26 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.98299999999999998,"15min":0.98099999999999998},"min":{"1min":0.71199999999999997,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2569999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.0760000000000001,"15min":1.0820000000000001},"min":{"1min":0.70199999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.2290000000000001,"5min":1.798,"15min":1.798},"last":1.153}]},{"osd":5,"last update":"Tue Mar 12 02:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.127,"15min":1.1399999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3109999999999999,"5min":1.554,"15min":1.554},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.095,"5min":1.113,"15min":1.1160000000000001},"min":{"1min":0.88400000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.3580000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.0589999999999999}]},{"osd":7,"last update":"Tue Mar 12 02:15:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.085}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T02:16:42.268 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-12T02:16:42.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:42 smithi153 bash[14340]: audit 2024-03-12T02:16:41.429611+0000 mgr.smithi153.asomxr (mgr.14195) 199 : audit 0 from='client.14466 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:16:42.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:42 smithi163 bash[22347]: audit 2024-03-12T02:16:41.429611+0000 mgr.smithi153.asomxr (mgr.14195) 199 : audit 0 from='client.14466 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:16:43.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:43 smithi163 bash[22347]: cluster 2024-03-12T02:16:42.377114+0000 mgr.smithi153.asomxr (mgr.14195) 200 : cluster 0 pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:44.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:43 smithi153 bash[14340]: cluster 2024-03-12T02:16:42.377114+0000 mgr.smithi153.asomxr (mgr.14195) 200 : cluster 0 pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:45.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:45 smithi163 bash[22347]: cluster 2024-03-12T02:16:44.377733+0000 mgr.smithi153.asomxr (mgr.14195) 201 : cluster 0 pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:45.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:45 smithi163 bash[22347]: audit 2024-03-12T02:16:45.377499+0000 mon.smithi153 (mon.0) 736 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:16:46.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:45 smithi153 bash[14340]: cluster 2024-03-12T02:16:44.377733+0000 mgr.smithi153.asomxr (mgr.14195) 201 : cluster 0 pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:46.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:45 smithi153 bash[14340]: audit 2024-03-12T02:16:45.377499+0000 mon.smithi153 (mon.0) 736 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:16:47.018 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:16:47.900 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:47 smithi153 bash[14340]: cluster 2024-03-12T02:16:46.378380+0000 mgr.smithi153.asomxr (mgr.14195) 202 : cluster 0 pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:47.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:47 smithi163 bash[22347]: cluster 2024-03-12T02:16:46.378380+0000 mgr.smithi153.asomxr (mgr.14195) 202 : cluster 0 pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:49.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:49 smithi163 bash[22347]: cluster 2024-03-12T02:16:48.379088+0000 mgr.smithi153.asomxr (mgr.14195) 203 : cluster 0 pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:50.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:49 smithi153 bash[14340]: cluster 2024-03-12T02:16:48.379088+0000 mgr.smithi153.asomxr (mgr.14195) 203 : cluster 0 pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:50.133 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:16:50.137 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2024-03-12T02:16:50.986 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-03-12T02:16:48.378724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":160,"num_read_kb":139,"num_write":269,"num_write_kb":5026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":749928448,"kb_used":221312,"kb_used_data":5128,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":749707136,"statfs":{"total":767926730752,"available":767700107264,"internally_reserved":0,"allocated":5251072,"data_stored":3181887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007706"},"pg_stats":[{"pgid":"1.0","version":"33'217","reported_seq":331,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-12T02:14:39.916835+0000","last_change":"2024-03-12T02:14:37.950323+0000","last_active":"2024-03-12T02:14:39.916835+0000","last_peered":"2024-03-12T02:14:39.916835+0000","last_clean":"2024-03-12T02:14:39.916835+0000","last_became_active":"2024-03-12T02:14:37.949936+0000","last_became_peered":"2024-03-12T02:14:37.949936+0000","last_unstale":"2024-03-12T02:14:39.916835+0000","last_undegraded":"2024-03-12T02:14:39.916835+0000","last_fullsized":"2024-03-12T02:14:39.916835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-12T02:14:14.924429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-12T02:14:14.924429+0000","last_clean_scrub_stamp":"2024-03-12T02:14:14.924429+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T09:47:15.243158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":160,"num_read_kb":139,"num_write":269,"num_write_kb":5026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":160,"num_read_kb":139,"num_write":269,"num_write_kb":5026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":3,"acting":3,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":39,"seq":167503724574,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713032,"statfs":{"total":95990841344,"available":95962144768,"internally_reserved":0,"allocated":1028096,"data_stored":766750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.85799999999999998,"15min":0.871},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1579999999999999,"5min":1.351,"15min":1.351},"last":0.753},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.84399999999999997,"15min":0.84899999999999998},"min":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.006,"5min":1.21,"15min":1.21},"last":0.69299999999999995}]},{"osd":1,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.35499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2649999999999999,"5min":1.365,"15min":1.365},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.58499999999999996}]},{"osd":2,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.88800000000000001,"15min":0.89400000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.97399999999999998,"15min":0.98799999999999999},"min":{"1min":0.621,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.2909999999999999,"5min":1.381,"15min":1.381},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.876,"15min":0.89100000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2629999999999999,"5min":1.659,"15min":1.659},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.90400000000000003,"15min":0.91700000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1639999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.90500000000000003}]},{"osd":4,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.97399999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1879999999999999,"5min":1.272,"15min":1.272},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0329999999999999,"15min":1.0489999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2270000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.82999999999999996}]},{"osd":5,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.91600000000000004,"15min":0.92800000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.96999999999999997,"15min":0.98399999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.872}]},{"osd":6,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.0669999999999999,"15min":1.085},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2450000000000001,"5min":1.488,"15min":1.488},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1020000000000001,"15min":1.1180000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3660000000000001,"5min":1.671,"15min":1.671},"last":1.04}]}]},{"osd":5,"up_from":35,"seq":150323855391,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713612,"statfs":{"total":95990841344,"available":95962738688,"internally_reserved":0,"allocated":434176,"data_stored":176382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.85999999999999999,"15min":0.87},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1,"5min":1.5,"15min":1.5},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.92100000000000004,"15min":0.93799999999999994},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2669999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.72899999999999998,"15min":0.74199999999999999},"min":{"1min":0.434,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.48899999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.115,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.86199999999999999,"15min":0.86599999999999999},"min":{"1min":0.623,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.89100000000000001,"15min":0.89200000000000002},"min":{"1min":0.63400000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.86699999999999999}]},{"osd":3,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.82199999999999995,"15min":0.83099999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1419999999999999,"5min":1.274,"15min":1.274},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.755,"15min":0.749},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.238,"5min":1.268,"15min":1.268},"last":0.5}]},{"osd":4,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.94599999999999995,"15min":0.94399999999999995},"min":{"1min":0.68500000000000005,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1699999999999999,"5min":1.325,"15min":1.325},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.016,"15min":1.024},"min":{"1min":0.78900000000000003,"5min":0.625,"15min":0.625},"max":{"1min":1.224,"5min":1.714,"15min":1.714},"last":0.98699999999999999}]},{"osd":6,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.0029999999999999,"15min":1.0069999999999999},"min":{"1min":0.77100000000000002,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.1619999999999999,"5min":1.494,"15min":1.494},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.97699999999999998,"15min":0.97599999999999998},"min":{"1min":0.73099999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.21,"5min":1.413,"15min":1.413},"last":0.91300000000000003}]},{"osd":7,"last update":"Tue Mar 12 02:16:39 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.97299999999999998,"15min":0.98899999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.171,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0489999999999999,"15min":1.0640000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3109999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.69699999999999995}]}]},{"osd":1,"up_from":20,"seq":85899345954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27444,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713612,"statfs":{"total":95990841344,"available":95962738688,"internally_reserved":0,"allocated":430080,"data_stored":176109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.91900000000000004,"15min":0.92900000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.127,"5min":1.321,"15min":1.321},"last":1.034},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.89600000000000002,"15min":0.89100000000000001},"min":{"1min":0.59799999999999998,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.1859999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.78500000000000003}]},{"osd":2,"last update":"Tue Mar 12 02:16:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84899999999999998,"15min":0.84699999999999998},"min":{"1min":0.51500000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0900000000000001,"5min":1.284,"15min":1.284},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.91200000000000003,"15min":0.91000000000000003},"min":{"1min":0.64800000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1080000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.94399999999999995}]},{"osd":3,"last update":"Tue Mar 12 02:16:23 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.77200000000000002,"15min":0.77300000000000002},"min":{"1min":0.46200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.434,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97099999999999997,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.74399999999999999}]},{"osd":4,"last update":"Tue Mar 12 02:16:23 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.92100000000000004,"15min":0.92700000000000005},"min":{"1min":0.66800000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.069,"5min":1.294,"15min":1.294},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1,"15min":1.01},"min":{"1min":0.65200000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.2370000000000001,"5min":1.359,"15min":1.359},"last":0.81799999999999995}]},{"osd":5,"last update":"Tue Mar 12 02:16:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.91300000000000003,"15min":0.93300000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3129999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.97499999999999998,"15min":0.996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1819999999999999,"5min":1.385,"15min":1.385},"last":1.0569999999999999}]},{"osd":6,"last update":"Tue Mar 12 02:16:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0960000000000001,"15min":1.1100000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4319999999999999,"5min":1.458,"15min":1.458},"last":1.113},{"interface":"front","average":{"1min":1.044,"5min":1.0860000000000001,"15min":1.093},"min":{"1min":0.69399999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.3029999999999999,"5min":1.484,"15min":1.484},"last":1.0029999999999999}]},{"osd":7,"last update":"Tue Mar 12 02:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.0760000000000001,"15min":1.095},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3360000000000001,"5min":1.387,"15min":1.387},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.081,"15min":1.097},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3899999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.079}]}]},{"osd":0,"up_from":16,"seq":68719476771,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713024,"statfs":{"total":95990841344,"available":95962136576,"internally_reserved":0,"allocated":1028096,"data_stored":766750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Mar 12 02:16:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81899999999999995,"15min":0.81699999999999995},"min":{"1min":0.58199999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.82299999999999995,"15min":0.81499999999999995},"min":{"1min":0.52500000000000002,"5min":0.499,"15min":0.499},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.94799999999999995}]},{"osd":2,"last update":"Tue Mar 12 02:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.70199999999999996,"15min":0.68700000000000006},"min":{"1min":0.54100000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.745},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.64600000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1479999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Mar 12 02:16:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91900000000000004,"15min":0.92700000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.1879999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.169}]},{"osd":4,"last update":"Tue Mar 12 02:16:24 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.94499999999999995,"15min":0.95299999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2310000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.88600000000000001,"15min":0.89200000000000002},"min":{"1min":0.59199999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.151,"5min":1.244,"15min":1.244},"last":0.81899999999999995}]},{"osd":5,"last update":"Tue Mar 12 02:16:34 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.0860000000000001,"15min":1.083},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":1.075},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.96399999999999997,"15min":0.96899999999999997},"min":{"1min":0.70299999999999996,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.1910000000000001,"5min":1.339,"15min":1.339},"last":1.103}]},{"osd":6,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.98299999999999998,"15min":0.98699999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.161,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0760000000000001,"15min":1.087},"min":{"1min":0.69699999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.1970000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.014}]},{"osd":7,"last update":"Tue Mar 12 02:16:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.107,"15min":1.1100000000000001},"min":{"1min":0.80200000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.3899999999999999,"5min":1.613,"15min":1.613},"last":1.198},{"interface":"front","average":{"1min":1.169,"5min":1.163,"15min":1.1619999999999999},"min":{"1min":0.92100000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.2290000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713604,"statfs":{"total":95990841344,"available":95962730496,"internally_reserved":0,"allocated":434176,"data_stored":176382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78100000000000003,"15min":0.77500000000000002},"min":{"1min":0.53700000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.999}]},{"osd":1,"last update":"Tue Mar 12 02:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.85799999999999998,"15min":0.86599999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.002,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.91300000000000003,"15min":0.91900000000000004},"min":{"1min":0.59699999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1699999999999999,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.1279999999999999}]},{"osd":3,"last update":"Tue Mar 12 02:16:23 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89700000000000002,"15min":0.89900000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.123,"5min":1.258,"15min":1.258},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.92100000000000004,"15min":0.92700000000000005},"min":{"1min":0.58599999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1439999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.81000000000000005}]},{"osd":4,"last update":"Tue Mar 12 02:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.92400000000000004,"15min":0.92700000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.103},{"interface":"front","average":{"1min":0.87,"5min":0.92700000000000005,"15min":0.93700000000000006},"min":{"1min":0.59699999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1439999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.1619999999999999}]},{"osd":5,"last update":"Tue Mar 12 02:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.038,"15min":1.0489999999999999},"min":{"1min":0.69899999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.2},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.014,"15min":1.0249999999999999},"min":{"1min":0.75800000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.2689999999999999,"5min":1.383,"15min":1.383},"last":1.2410000000000001}]},{"osd":6,"last update":"Tue Mar 12 02:16:23 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.98799999999999999,"15min":0.97999999999999998},"min":{"1min":0.83399999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.2450000000000001,"5min":1.339,"15min":1.339},"last":1.075},{"interface":"front","average":{"1min":1.099,"5min":1.0580000000000001,"15min":1.0509999999999999},"min":{"1min":0.86399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.1850000000000001}]},{"osd":7,"last update":"Tue Mar 12 02:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.2609999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1499999999999999,"15min":1.153},"min":{"1min":0.72899999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5209999999999999,"5min":1.649,"15min":1.649},"last":1.2669999999999999}]}]},{"osd":4,"up_from":26,"seq":111669149729,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713612,"statfs":{"total":95990841344,"available":95962738688,"internally_reserved":0,"allocated":434176,"data_stored":176382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:20 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.54600000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77000000000000002,"15min":0.76200000000000001},"min":{"1min":0.66400000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.95999999999999996}]},{"osd":1,"last update":"Tue Mar 12 02:16:20 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.879,"15min":0.873},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3029999999999999,"5min":1.391,"15min":1.391},"last":1.034},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.99299999999999999}]},{"osd":2,"last update":"Tue Mar 12 02:16:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77600000000000002,"15min":0.77200000000000002},"min":{"1min":0.60699999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.875,"15min":0.86399999999999999},"min":{"1min":0.71599999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.2270000000000001,"5min":1.306,"15min":1.306},"last":0.75}]},{"osd":3,"last update":"Tue Mar 12 02:16:25 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.96399999999999997,"15min":0.95599999999999996},"min":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.1819999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.98999999999999999,"15min":0.99399999999999999},"min":{"1min":0.75700000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.2350000000000001,"5min":1.353,"15min":1.353},"last":1.0800000000000001}]},{"osd":5,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0469999999999999,"15min":1.044},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.0740000000000001,"15min":1.0780000000000001},"min":{"1min":0.84499999999999997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.3480000000000001,"5min":1.47,"15min":1.47},"last":1.169}]},{"osd":6,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0880000000000001,"15min":1.0900000000000001},"min":{"1min":0.83199999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.381,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0649999999999999,"15min":1.0580000000000001},"min":{"1min":0.79800000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.502,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.2150000000000001}]},{"osd":7,"last update":"Tue Mar 12 02:16:35 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1619999999999999,"15min":1.159},"min":{"1min":0.878,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.149,"15min":1.151},"min":{"1min":0.95199999999999996,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.2669999999999999}]}]},{"osd":3,"up_from":29,"seq":124554051617,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713028,"statfs":{"total":95990841344,"available":95962140672,"internally_reserved":0,"allocated":1028096,"data_stored":766750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.82399999999999995,"15min":0.83399999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98399999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.90600000000000003,"15min":0.91600000000000004},"min":{"1min":0.55600000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.129,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68999999999999995,"15min":0.69899999999999995},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.02,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.79400000000000004,"15min":0.80600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.018,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.92000000000000004}]},{"osd":2,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86799999999999999,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.92900000000000005,"15min":0.93100000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.202,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.629}]},{"osd":4,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.3,"5min":1.387,"15min":1.387},"last":1.002},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0269999999999999,"15min":1.034},"min":{"1min":0.63700000000000001,"5min":0.436,"15min":0.436},"max":{"1min":1.1579999999999999,"5min":1.587,"15min":1.587},"last":0.752}]},{"osd":5,"last update":"Tue Mar 12 02:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.93300000000000005,"15min":0.95399999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.3180000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.97399999999999998,"15min":1.002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3660000000000001,"5min":1.468,"15min":1.468},"last":0.80200000000000005}]},{"osd":6,"last update":"Tue Mar 12 02:16:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.117,"15min":1.1379999999999999},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.2609999999999999,"5min":1.52,"15min":1.52},"last":1.075},{"interface":"front","average":{"1min":1.034,"5min":1.119,"15min":1.133},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.5069999999999999,"5min":1.532,"15min":1.532},"last":0.68400000000000005}]},{"osd":7,"last update":"Tue Mar 12 02:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0349999999999999,"15min":1.0640000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.258,"5min":1.448,"15min":1.448},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.1220000000000001,"15min":1.1479999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4199999999999999,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.65700000000000003}]}]},{"osd":6,"up_from":31,"seq":133143986208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93741056,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713612,"statfs":{"total":95990841344,"available":95962738688,"internally_reserved":0,"allocated":434176,"data_stored":176382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Mar 12 02:16:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81899999999999995,"15min":0.81799999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.325,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Tue Mar 12 02:16:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86499999999999999,"15min":0.872},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.03,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.871},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89300000000000002,"15min":0.89400000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1080000000000001,"5min":1.131,"15min":1.131},"last":0.89700000000000002}]},{"osd":2,"last update":"Tue Mar 12 02:16:26 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85699999999999998,"15min":0.84599999999999997},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.91600000000000004,"15min":0.92400000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2270000000000001,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Tue Mar 12 02:16:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.98399999999999999,"15min":0.996},"min":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.1020000000000001,"5min":1.627,"15min":1.627},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.999,"15min":1.0109999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.1899999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.97099999999999997}]},{"osd":4,"last update":"Tue Mar 12 02:16:26 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.98299999999999998,"15min":0.98099999999999998},"min":{"1min":0.71199999999999997,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2569999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.044,"5min":1.0760000000000001,"15min":1.0820000000000001},"min":{"1min":0.70199999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.2290000000000001,"5min":1.798,"15min":1.798},"last":0.81499999999999995}]},{"osd":5,"last update":"Tue Mar 12 02:16:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.127,"15min":1.1399999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3109999999999999,"5min":1.554,"15min":1.554},"last":1.002},{"interface":"front","average":{"1min":1.095,"5min":1.113,"15min":1.1160000000000001},"min":{"1min":0.88400000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.3580000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.101}]},{"osd":7,"last update":"Tue Mar 12 02:16:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1040000000000001,"15min":1.101},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.3129999999999999,"5min":1.387,"15min":1.387},"last":0.75},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1819999999999999,"15min":1.1890000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.375,"5min":1.613,"15min":1.613},"last":1.069}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-12T02:16:50.987 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-12T02:16:50.988 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-12T02:16:50.988 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-12T02:16:50.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph health --format=json 2024-03-12T02:16:51.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:51 smithi163 bash[22347]: audit 2024-03-12T02:16:50.122323+0000 mgr.smithi153.asomxr (mgr.14195) 204 : audit 0 from='client.14470 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:16:51.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:51 smithi163 bash[22347]: cluster 2024-03-12T02:16:50.379728+0000 mgr.smithi153.asomxr (mgr.14195) 205 : cluster 0 pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:52.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:51 smithi153 bash[14340]: audit 2024-03-12T02:16:50.122323+0000 mgr.smithi153.asomxr (mgr.14195) 204 : audit 0 from='client.14470 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:16:52.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:51 smithi153 bash[14340]: cluster 2024-03-12T02:16:50.379728+0000 mgr.smithi153.asomxr (mgr.14195) 205 : cluster 0 pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:53.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:53 smithi163 bash[22347]: cluster 2024-03-12T02:16:52.380287+0000 mgr.smithi153.asomxr (mgr.14195) 206 : cluster 0 pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:54.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:53 smithi153 bash[14340]: cluster 2024-03-12T02:16:52.380287+0000 mgr.smithi153.asomxr (mgr.14195) 206 : cluster 0 pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:55.733 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:16:55.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:55 smithi163 bash[22347]: cluster 2024-03-12T02:16:54.382652+0000 mgr.smithi153.asomxr (mgr.14195) 207 : cluster 0 pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:56.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:55 smithi153 bash[14340]: cluster 2024-03-12T02:16:54.382652+0000 mgr.smithi153.asomxr (mgr.14195) 207 : cluster 0 pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:57.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:57 smithi163 bash[22347]: cluster 2024-03-12T02:16:56.383038+0000 mgr.smithi153.asomxr (mgr.14195) 208 : cluster 0 pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:58.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:57 smithi153 bash[14340]: cluster 2024-03-12T02:16:56.383038+0000 mgr.smithi153.asomxr (mgr.14195) 208 : cluster 0 pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:58.909 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:16:58.909 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-12T02:16:59.622 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-12T02:16:59.622 INFO:tasks.cephadm:Setup complete, yielding 2024-03-12T02:16:59.622 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-12T02:16:59.632 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-12T02:16:59.633 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- bash -c 'ceph orch status' 2024-03-12T02:16:59.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:59 smithi163 bash[22347]: cluster 2024-03-12T02:16:58.383676+0000 mgr.smithi153.asomxr (mgr.14195) 209 : cluster 0 pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:16:59.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:16:59 smithi163 bash[22347]: audit 2024-03-12T02:16:58.905737+0000 mon.smithi153 (mon.0) 737 : audit 0 from='client.? 172.21.15.153:0/3589924511' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-12T02:17:00.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:59 smithi153 bash[14340]: cluster 2024-03-12T02:16:58.383676+0000 mgr.smithi153.asomxr (mgr.14195) 209 : cluster 0 pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:00.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:16:59 smithi153 bash[14340]: audit 2024-03-12T02:16:58.905737+0000 mon.smithi153 (mon.0) 737 : audit 0 from='client.? 172.21.15.153:0/3589924511' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-12T02:17:00.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:00 smithi163 bash[22347]: audit 2024-03-12T02:17:00.378041+0000 mon.smithi153 (mon.0) 738 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:17:01.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:00 smithi153 bash[14340]: audit 2024-03-12T02:17:00.378041+0000 mon.smithi153 (mon.0) 738 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:17:01.979 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:01 smithi163 bash[22347]: cluster 2024-03-12T02:17:00.384738+0000 mgr.smithi153.asomxr (mgr.14195) 210 : cluster 0 pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:01.997 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:01 smithi153 bash[14340]: cluster 2024-03-12T02:17:00.384738+0000 mgr.smithi153.asomxr (mgr.14195) 210 : cluster 0 pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:03.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:03 smithi163 bash[22347]: cluster 2024-03-12T02:17:02.385406+0000 mgr.smithi153.asomxr (mgr.14195) 211 : cluster 0 pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:04.007 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:03 smithi153 bash[14340]: cluster 2024-03-12T02:17:02.385406+0000 mgr.smithi153.asomxr (mgr.14195) 211 : cluster 0 pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:04.378 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:17:05.940 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:05 smithi163 bash[22347]: cluster 2024-03-12T02:17:04.385867+0000 mgr.smithi153.asomxr (mgr.14195) 212 : cluster 0 pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:06.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:05 smithi153 bash[14340]: cluster 2024-03-12T02:17:04.385867+0000 mgr.smithi153.asomxr (mgr.14195) 212 : cluster 0 pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:07.580 INFO:teuthology.orchestra.run.smithi153.stdout:Backend: cephadm 2024-03-12T02:17:07.580 INFO:teuthology.orchestra.run.smithi153.stdout:Available: Yes 2024-03-12T02:17:07.580 INFO:teuthology.orchestra.run.smithi153.stdout:Paused: No 2024-03-12T02:17:07.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:07 smithi163 bash[22347]: cluster 2024-03-12T02:17:06.386362+0000 mgr.smithi153.asomxr (mgr.14195) 213 : cluster 0 pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:08.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:07 smithi153 bash[14340]: cluster 2024-03-12T02:17:06.386362+0000 mgr.smithi153.asomxr (mgr.14195) 213 : cluster 0 pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:08.315 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-12T02:17:08.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:08 smithi163 bash[22347]: audit 2024-03-12T02:17:07.576273+0000 mgr.smithi153.asomxr (mgr.14195) 214 : audit 0 from='client.14478 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:09.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:08 smithi153 bash[14340]: audit 2024-03-12T02:17:07.576273+0000 mgr.smithi153.asomxr (mgr.14195) 214 : audit 0 from='client.14478 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:09.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:09 smithi163 bash[22347]: cluster 2024-03-12T02:17:08.386803+0000 mgr.smithi153.asomxr (mgr.14195) 215 : cluster 0 pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:10.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:09 smithi153 bash[14340]: cluster 2024-03-12T02:17:08.386803+0000 mgr.smithi153.asomxr (mgr.14195) 215 : cluster 0 pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:11.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:11 smithi163 bash[22347]: cluster 2024-03-12T02:17:10.387357+0000 mgr.smithi153.asomxr (mgr.14195) 216 : cluster 0 pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:12.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:11 smithi153 bash[14340]: cluster 2024-03-12T02:17:10.387357+0000 mgr.smithi153.asomxr (mgr.14195) 216 : cluster 0 pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:13.056 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:17:13.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:13 smithi163 bash[22347]: cluster 2024-03-12T02:17:12.387934+0000 mgr.smithi153.asomxr (mgr.14195) 217 : cluster 0 pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:13.987 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:13 smithi153 bash[14340]: cluster 2024-03-12T02:17:12.387934+0000 mgr.smithi153.asomxr (mgr.14195) 217 : cluster 0 pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:15.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:15 smithi163 bash[22347]: cluster 2024-03-12T02:17:14.388402+0000 mgr.smithi153.asomxr (mgr.14195) 218 : cluster 0 pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:15.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:15 smithi163 bash[22347]: audit 2024-03-12T02:17:15.377662+0000 mon.smithi153 (mon.0) 739 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:17:16.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:15 smithi153 bash[14340]: cluster 2024-03-12T02:17:14.388402+0000 mgr.smithi153.asomxr (mgr.14195) 218 : cluster 0 pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:16.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:15 smithi153 bash[14340]: audit 2024-03-12T02:17:15.377662+0000 mon.smithi153 (mon.0) 739 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:17:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-12T02:17:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (4m) 37s ago 6m 13.9M - 0.25.0 c8568f914cd2 51b44834af1d 2024-03-12T02:17:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (6m) 37s ago 6m 7872k - 18.2.1-746-gc2ea977b a30af69eda22 284c95677e24 2024-03-12T02:17:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi163 smithi163 running (5m) 48s ago 5m 8076k - 18.2.1-746-gc2ea977b a30af69eda22 326a07f6b216 2024-03-12T02:17:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (6m) 37s ago 6m 7123k - 18.2.1-746-gc2ea977b a30af69eda22 29c68793fc7f 2024-03-12T02:17:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi163 smithi163 running (4m) 48s ago 4m 7096k - 18.2.1-746-gc2ea977b a30af69eda22 a2949c63c06b 2024-03-12T02:17:16.242 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (4m) 37s ago 5m 76.9M - 9.4.7 954c08fa6188 8e911cf9185c 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.asomxr smithi153 *:9283,8765,8443 running (8m) 37s ago 8m 483M - 18.2.1-746-gc2ea977b a30af69eda22 ed74783bede6 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi163.fvmiae smithi163 *:8443,9283,8765 running (4m) 48s ago 4m 434M - 18.2.1-746-gc2ea977b a30af69eda22 eb062a46b56a 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (8m) 37s ago 8m 41.8M 2048M 18.2.1-746-gc2ea977b a30af69eda22 01a04f20f7f2 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi163 smithi163 running (4m) 48s ago 4m 32.3M 2048M 18.2.1-746-gc2ea977b a30af69eda22 31098bfaf49f 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (6m) 37s ago 6m 8307k - 1.5.0 0da6a335fe13 d82dd8a4da36 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi163 smithi163 *:9100 running (4m) 48s ago 4m 8156k - 1.5.0 0da6a335fe13 2508195ef5ba 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi163 running (3m) 48s ago 3m 41.2M 3772M 18.2.1-746-gc2ea977b a30af69eda22 1bf481d10aa8 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (3m) 37s ago 3m 39.9M 3004M 18.2.1-746-gc2ea977b a30af69eda22 c5302a09a94e 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi163 running (3m) 48s ago 3m 38.5M 3772M 18.2.1-746-gc2ea977b a30af69eda22 cd32e60ee108 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi153 running (3m) 37s ago 3m 39.0M 3004M 18.2.1-746-gc2ea977b a30af69eda22 a73a4b706ab1 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi163 running (3m) 48s ago 3m 39.0M 3772M 18.2.1-746-gc2ea977b a30af69eda22 c5c32704f39c 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi153 running (2m) 37s ago 2m 36.4M 3004M 18.2.1-746-gc2ea977b a30af69eda22 905af52987fe 2024-03-12T02:17:16.243 INFO:teuthology.orchestra.run.smithi153.stdout:osd.6 smithi163 running (3m) 48s ago 3m 36.1M 3772M 18.2.1-746-gc2ea977b a30af69eda22 7eef3e567643 2024-03-12T02:17:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:osd.7 smithi153 running (2m) 37s ago 2m 39.1M 3004M 18.2.1-746-gc2ea977b a30af69eda22 64751e9e9573 2024-03-12T02:17:16.244 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (4m) 37s ago 5m 46.8M - 2.43.0 a07b618ecd1d cd351e2ab70b 2024-03-12T02:17:17.088 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-12T02:17:17.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:17 smithi163 bash[22347]: audit 2024-03-12T02:17:16.218984+0000 mgr.smithi153.asomxr (mgr.14195) 219 : audit 0 from='client.14482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:17.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:17 smithi163 bash[22347]: cluster 2024-03-12T02:17:16.388925+0000 mgr.smithi153.asomxr (mgr.14195) 220 : cluster 0 pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:18.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:17 smithi153 bash[14340]: audit 2024-03-12T02:17:16.218984+0000 mgr.smithi153.asomxr (mgr.14195) 219 : audit 0 from='client.14482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:18.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:17 smithi153 bash[14340]: cluster 2024-03-12T02:17:16.388925+0000 mgr.smithi153.asomxr (mgr.14195) 220 : cluster 0 pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:19.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:19 smithi163 bash[22347]: cluster 2024-03-12T02:17:18.389517+0000 mgr.smithi153.asomxr (mgr.14195) 221 : cluster 0 pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:20.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:19 smithi153 bash[14340]: cluster 2024-03-12T02:17:18.389517+0000 mgr.smithi153.asomxr (mgr.14195) 221 : cluster 0 pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:21.831 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:17:21.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:21 smithi163 bash[22347]: cluster 2024-03-12T02:17:20.390065+0000 mgr.smithi153.asomxr (mgr.14195) 222 : cluster 0 pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:22.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:21 smithi153 bash[14340]: cluster 2024-03-12T02:17:20.390065+0000 mgr.smithi153.asomxr (mgr.14195) 222 : cluster 0 pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:23.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:23 smithi163 bash[22347]: cluster 2024-03-12T02:17:22.390656+0000 mgr.smithi153.asomxr (mgr.14195) 223 : cluster 0 pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:24.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:23 smithi153 bash[14340]: cluster 2024-03-12T02:17:22.390656+0000 mgr.smithi153.asomxr (mgr.14195) 223 : cluster 0 pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:24.929 INFO:teuthology.orchestra.run.smithi153.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-12T02:17:24.930 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager ?:9093,9094 1/1 46s ago 7m count:1 2024-03-12T02:17:24.930 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter 2/2 56s ago 7m * 2024-03-12T02:17:24.930 INFO:teuthology.orchestra.run.smithi153.stdout:crash 2/2 56s ago 7m * 2024-03-12T02:17:24.930 INFO:teuthology.orchestra.run.smithi153.stdout:grafana ?:3000 1/1 46s ago 7m count:1 2024-03-12T02:17:24.930 INFO:teuthology.orchestra.run.smithi153.stdout:mgr 2/2 56s ago 7m count:2 2024-03-12T02:17:24.930 INFO:teuthology.orchestra.run.smithi153.stdout:mon 2/2 56s ago 5m smithi153:172.21.15.153=smithi153;smithi163:172.21.15.163=smithi163;count:2 2024-03-12T02:17:24.930 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter ?:9100 2/2 56s ago 7m * 2024-03-12T02:17:24.930 INFO:teuthology.orchestra.run.smithi153.stdout:osd.all-available-devices 8 56s ago 4m * 2024-03-12T02:17:24.930 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus ?:9095 1/1 46s ago 7m count:1 2024-03-12T02:17:25.712 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-12T02:17:25.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:25 smithi163 bash[22347]: cluster 2024-03-12T02:17:24.391085+0000 mgr.smithi153.asomxr (mgr.14195) 224 : cluster 0 pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:25.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:25 smithi163 bash[22347]: audit 2024-03-12T02:17:24.919820+0000 mgr.smithi153.asomxr (mgr.14195) 225 : audit 0 from='client.14486 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:26.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:25 smithi153 bash[14340]: cluster 2024-03-12T02:17:24.391085+0000 mgr.smithi153.asomxr (mgr.14195) 224 : cluster 0 pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:26.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:25 smithi153 bash[14340]: audit 2024-03-12T02:17:24.919820+0000 mgr.smithi153.asomxr (mgr.14195) 225 : audit 0 from='client.14486 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:27.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:27 smithi163 bash[22347]: cluster 2024-03-12T02:17:26.391622+0000 mgr.smithi153.asomxr (mgr.14195) 226 : cluster 0 pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:28.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:27 smithi153 bash[14340]: cluster 2024-03-12T02:17:26.391622+0000 mgr.smithi153.asomxr (mgr.14195) 226 : cluster 0 pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:29.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:29 smithi153 bash[14340]: cluster 2024-03-12T02:17:28.392198+0000 mgr.smithi153.asomxr (mgr.14195) 227 : cluster 0 pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:29.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:29 smithi163 bash[22347]: cluster 2024-03-12T02:17:28.392198+0000 mgr.smithi153.asomxr (mgr.14195) 227 : cluster 0 pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:30.457 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:17:30.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:30 smithi163 bash[22347]: audit 2024-03-12T02:17:30.378090+0000 mon.smithi153 (mon.0) 740 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:17:31.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:30 smithi153 bash[14340]: audit 2024-03-12T02:17:30.378090+0000 mon.smithi153 (mon.0) 740 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:17:31.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:31 smithi163 bash[22347]: cluster 2024-03-12T02:17:30.392737+0000 mgr.smithi153.asomxr (mgr.14195) 228 : cluster 0 pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:32.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:31 smithi153 bash[14340]: cluster 2024-03-12T02:17:30.392737+0000 mgr.smithi153.asomxr (mgr.14195) 228 : cluster 0 pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:33.734 INFO:teuthology.orchestra.run.smithi153.stdout:HOST ADDR LABELS STATUS 2024-03-12T02:17:33.734 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153 172.21.15.153 2024-03-12T02:17:33.734 INFO:teuthology.orchestra.run.smithi153.stdout:smithi163 172.21.15.163 2024-03-12T02:17:33.734 INFO:teuthology.orchestra.run.smithi153.stdout:2 hosts in cluster 2024-03-12T02:17:34.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:33 smithi153 bash[14340]: cluster 2024-03-12T02:17:32.393313+0000 mgr.smithi153.asomxr (mgr.14195) 229 : cluster 0 pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:34.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:33 smithi163 bash[22347]: cluster 2024-03-12T02:17:32.393313+0000 mgr.smithi153.asomxr (mgr.14195) 229 : cluster 0 pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:34.552 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-12T02:17:34.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:34 smithi153 bash[14340]: audit 2024-03-12T02:17:33.728760+0000 mgr.smithi153.asomxr (mgr.14195) 230 : audit 0 from='client.14490 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:35.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:34 smithi163 bash[22347]: audit 2024-03-12T02:17:33.728760+0000 mgr.smithi153.asomxr (mgr.14195) 230 : audit 0 from='client.14490 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:36.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:35 smithi153 bash[14340]: cluster 2024-03-12T02:17:34.393863+0000 mgr.smithi153.asomxr (mgr.14195) 231 : cluster 0 pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:36.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:35 smithi163 bash[22347]: cluster 2024-03-12T02:17:34.393863+0000 mgr.smithi153.asomxr (mgr.14195) 231 : cluster 0 pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:38.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:37 smithi153 bash[14340]: cluster 2024-03-12T02:17:36.394461+0000 mgr.smithi153.asomxr (mgr.14195) 232 : cluster 0 pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:38.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:37 smithi163 bash[22347]: cluster 2024-03-12T02:17:36.394461+0000 mgr.smithi153.asomxr (mgr.14195) 232 : cluster 0 pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:39.297 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:17:40.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:39 smithi153 bash[14340]: cluster 2024-03-12T02:17:38.395066+0000 mgr.smithi153.asomxr (mgr.14195) 233 : cluster 0 pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:40.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:39 smithi153 bash[14340]: audit 2024-03-12T02:17:39.074704+0000 mon.smithi153 (mon.0) 741 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:17:40.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:39 smithi163 bash[22347]: cluster 2024-03-12T02:17:38.395066+0000 mgr.smithi153.asomxr (mgr.14195) 233 : cluster 0 pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:40.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:39 smithi163 bash[22347]: audit 2024-03-12T02:17:39.074704+0000 mon.smithi153 (mon.0) 741 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:17:41.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:41 smithi163 bash[22347]: cluster 2024-03-12T02:17:40.395684+0000 mgr.smithi153.asomxr (mgr.14195) 234 : cluster 0 pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:42.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:41 smithi153 bash[14340]: cluster 2024-03-12T02:17:40.395684+0000 mgr.smithi153.asomxr (mgr.14195) 234 : cluster 0 pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:42.602 INFO:teuthology.orchestra.run.smithi153.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-12T02:17:42.602 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300MV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-12T02:17:42.602 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153 /dev/nvme1n1 ssd Linux_66cad7e9715b5346e918 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-03-12T02:17:42.602 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153 /dev/nvme2n1 ssd Linux_2d3ee23c267222953e9a 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-03-12T02:17:42.603 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153 /dev/nvme3n1 ssd Linux_ff4d93fbcd80f7918942 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-03-12T02:17:42.603 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153 /dev/nvme4n1 ssd Linux_09d4539be4bf518a66af 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-03-12T02:17:42.603 INFO:teuthology.orchestra.run.smithi153.stdout:smithi163 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GU400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-12T02:17:42.603 INFO:teuthology.orchestra.run.smithi153.stdout:smithi163 /dev/nvme1n1 ssd Linux_5fc5180e4fa8982c5e82 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-03-12T02:17:42.603 INFO:teuthology.orchestra.run.smithi153.stdout:smithi163 /dev/nvme2n1 ssd Linux_ab937578e92804d56195 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-03-12T02:17:42.603 INFO:teuthology.orchestra.run.smithi153.stdout:smithi163 /dev/nvme3n1 ssd Linux_5d5434f9542ba7a370ce 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-03-12T02:17:42.603 INFO:teuthology.orchestra.run.smithi153.stdout:smithi163 /dev/nvme4n1 ssd Linux_550a77cac533c9459986 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-03-12T02:17:43.471 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-12T02:17:43.480 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-12T02:17:43.481 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- bash -c 'ceph orch apply jaeger' 2024-03-12T02:17:44.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:43 smithi153 bash[14340]: cluster 2024-03-12T02:17:42.396131+0000 mgr.smithi153.asomxr (mgr.14195) 235 : cluster 0 pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:44.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:43 smithi153 bash[14340]: audit 2024-03-12T02:17:42.590074+0000 mgr.smithi153.asomxr (mgr.14195) 236 : audit 0 from='client.14494 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:44.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:43 smithi163 bash[22347]: cluster 2024-03-12T02:17:42.396131+0000 mgr.smithi153.asomxr (mgr.14195) 235 : cluster 0 pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:44.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:43 smithi163 bash[22347]: audit 2024-03-12T02:17:42.590074+0000 mgr.smithi153.asomxr (mgr.14195) 236 : audit 0 from='client.14494 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:45.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:45 smithi163 bash[22347]: cluster 2024-03-12T02:17:44.396799+0000 mgr.smithi153.asomxr (mgr.14195) 237 : cluster 0 pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:45.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:45 smithi163 bash[22347]: audit 2024-03-12T02:17:44.665728+0000 mon.smithi153 (mon.0) 742 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:45.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:45 smithi163 bash[22347]: audit 2024-03-12T02:17:44.672957+0000 mon.smithi153 (mon.0) 743 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:45.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:45 smithi163 bash[22347]: audit 2024-03-12T02:17:45.378848+0000 mon.smithi153 (mon.0) 744 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:17:45.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:45 smithi163 bash[22347]: audit 2024-03-12T02:17:45.383336+0000 mon.smithi153 (mon.0) 745 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:45.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:45 smithi163 bash[22347]: audit 2024-03-12T02:17:45.392285+0000 mon.smithi153 (mon.0) 746 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:46.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:45 smithi153 bash[14340]: cluster 2024-03-12T02:17:44.396799+0000 mgr.smithi153.asomxr (mgr.14195) 237 : cluster 0 pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:46.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:45 smithi153 bash[14340]: audit 2024-03-12T02:17:44.665728+0000 mon.smithi153 (mon.0) 742 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:46.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:45 smithi153 bash[14340]: audit 2024-03-12T02:17:44.672957+0000 mon.smithi153 (mon.0) 743 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:46.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:45 smithi153 bash[14340]: audit 2024-03-12T02:17:45.378848+0000 mon.smithi153 (mon.0) 744 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:17:46.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:45 smithi153 bash[14340]: audit 2024-03-12T02:17:45.383336+0000 mon.smithi153 (mon.0) 745 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:46.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:45 smithi153 bash[14340]: audit 2024-03-12T02:17:45.392285+0000 mon.smithi153 (mon.0) 746 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:46.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:46 smithi163 bash[22347]: audit 2024-03-12T02:17:45.871897+0000 mon.smithi153 (mon.0) 747 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:17:46.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:46 smithi163 bash[22347]: audit 2024-03-12T02:17:45.873900+0000 mon.smithi153 (mon.0) 748 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:17:46.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:46 smithi163 bash[22347]: audit 2024-03-12T02:17:45.882095+0000 mon.smithi153 (mon.0) 749 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:46.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:46 smithi163 bash[22347]: audit 2024-03-12T02:17:45.886937+0000 mon.smithi153 (mon.0) 750 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:17:47.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:46 smithi153 bash[14340]: audit 2024-03-12T02:17:45.871897+0000 mon.smithi153 (mon.0) 747 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:17:47.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:46 smithi153 bash[14340]: audit 2024-03-12T02:17:45.873900+0000 mon.smithi153 (mon.0) 748 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:17:47.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:46 smithi153 bash[14340]: audit 2024-03-12T02:17:45.882095+0000 mon.smithi153 (mon.0) 749 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:47.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:46 smithi153 bash[14340]: audit 2024-03-12T02:17:45.886937+0000 mon.smithi153 (mon.0) 750 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:17:48.032 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:17:48.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:47 smithi163 bash[22347]: cluster 2024-03-12T02:17:46.397419+0000 mgr.smithi153.asomxr (mgr.14195) 238 : cluster 0 pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:48.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:47 smithi153 bash[14340]: cluster 2024-03-12T02:17:46.397419+0000 mgr.smithi153.asomxr (mgr.14195) 238 : cluster 0 pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:50.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:49 smithi163 bash[22347]: cluster 2024-03-12T02:17:48.398071+0000 mgr.smithi153.asomxr (mgr.14195) 239 : cluster 0 pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:50.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:49 smithi153 bash[14340]: cluster 2024-03-12T02:17:48.398071+0000 mgr.smithi153.asomxr (mgr.14195) 239 : cluster 0 pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:51.373 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled elasticsearch update... 2024-03-12T02:17:51.373 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled jaeger-collector update... 2024-03-12T02:17:51.373 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled jaeger-query update... 2024-03-12T02:17:51.373 INFO:teuthology.orchestra.run.smithi153.stdout:Scheduled jaeger-agent update... 2024-03-12T02:17:52.204 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:51 smithi153 bash[14340]: cluster 2024-03-12T02:17:50.398612+0000 mgr.smithi153.asomxr (mgr.14195) 240 : cluster 0 pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:52.204 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:51 smithi153 bash[14340]: audit 2024-03-12T02:17:51.354857+0000 mon.smithi153 (mon.0) 751 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:52.204 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:51 smithi153 bash[14340]: audit 2024-03-12T02:17:51.360147+0000 mon.smithi153 (mon.0) 752 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:52.204 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:51 smithi153 bash[14340]: audit 2024-03-12T02:17:51.364754+0000 mon.smithi153 (mon.0) 753 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:52.204 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:51 smithi153 bash[14340]: audit 2024-03-12T02:17:51.369300+0000 mon.smithi153 (mon.0) 754 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:52.204 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:51 smithi153 bash[14340]: audit 2024-03-12T02:17:51.403265+0000 mon.smithi153 (mon.0) 755 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:17:52.205 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service... 2024-03-12T02:17:52.214 INFO:tasks.cephadm:Waiting for ceph service elasticsearch to start (timeout 300)... 2024-03-12T02:17:52.215 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:17:52.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:51 smithi163 bash[22347]: cluster 2024-03-12T02:17:50.398612+0000 mgr.smithi153.asomxr (mgr.14195) 240 : cluster 0 pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:52.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:51 smithi163 bash[22347]: audit 2024-03-12T02:17:51.354857+0000 mon.smithi153 (mon.0) 751 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:52.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:51 smithi163 bash[22347]: audit 2024-03-12T02:17:51.360147+0000 mon.smithi153 (mon.0) 752 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:52.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:51 smithi163 bash[22347]: audit 2024-03-12T02:17:51.364754+0000 mon.smithi153 (mon.0) 753 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:52.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:51 smithi163 bash[22347]: audit 2024-03-12T02:17:51.369300+0000 mon.smithi153 (mon.0) 754 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:52.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:51 smithi163 bash[22347]: audit 2024-03-12T02:17:51.403265+0000 mon.smithi153 (mon.0) 755 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:17:53.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:52 smithi163 bash[22347]: audit 2024-03-12T02:17:51.346800+0000 mgr.smithi153.asomxr (mgr.14195) 241 : audit 0 from='client.14498 -' entity='client.admin' cmd=[{"prefix": "orch apply jaeger", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:53.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:52 smithi163 bash[22347]: cephadm 2024-03-12T02:17:51.348721+0000 mgr.smithi153.asomxr (mgr.14195) 242 : cephadm 1 Saving service elasticsearch spec with placement count:1 2024-03-12T02:17:53.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:52 smithi163 bash[22347]: cephadm 2024-03-12T02:17:51.355645+0000 mgr.smithi153.asomxr (mgr.14195) 243 : cephadm 1 Saving service jaeger-collector spec with placement count:1 2024-03-12T02:17:53.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:52 smithi163 bash[22347]: cephadm 2024-03-12T02:17:51.360894+0000 mgr.smithi153.asomxr (mgr.14195) 244 : cephadm 1 Saving service jaeger-query spec with placement count:1 2024-03-12T02:17:53.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:52 smithi163 bash[22347]: cephadm 2024-03-12T02:17:51.365600+0000 mgr.smithi153.asomxr (mgr.14195) 245 : cephadm 1 Saving service jaeger-agent spec with placement * 2024-03-12T02:17:53.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:52 smithi153 bash[14340]: audit 2024-03-12T02:17:51.346800+0000 mgr.smithi153.asomxr (mgr.14195) 241 : audit 0 from='client.14498 -' entity='client.admin' cmd=[{"prefix": "orch apply jaeger", "target": ["mon-mgr", ""]}]: dispatch 2024-03-12T02:17:53.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:52 smithi153 bash[14340]: cephadm 2024-03-12T02:17:51.348721+0000 mgr.smithi153.asomxr (mgr.14195) 242 : cephadm 1 Saving service elasticsearch spec with placement count:1 2024-03-12T02:17:53.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:52 smithi153 bash[14340]: cephadm 2024-03-12T02:17:51.355645+0000 mgr.smithi153.asomxr (mgr.14195) 243 : cephadm 1 Saving service jaeger-collector spec with placement count:1 2024-03-12T02:17:53.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:52 smithi153 bash[14340]: cephadm 2024-03-12T02:17:51.360894+0000 mgr.smithi153.asomxr (mgr.14195) 244 : cephadm 1 Saving service jaeger-query spec with placement count:1 2024-03-12T02:17:53.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:52 smithi153 bash[14340]: cephadm 2024-03-12T02:17:51.365600+0000 mgr.smithi153.asomxr (mgr.14195) 245 : cephadm 1 Saving service jaeger-agent spec with placement * 2024-03-12T02:17:54.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:53 smithi163 bash[22347]: cluster 2024-03-12T02:17:52.399115+0000 mgr.smithi153.asomxr (mgr.14195) 246 : cluster 0 pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:54.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:53 smithi153 bash[14340]: cluster 2024-03-12T02:17:52.399115+0000 mgr.smithi153.asomxr (mgr.14195) 246 : cluster 0 pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:56.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:55 smithi163 bash[22347]: cluster 2024-03-12T02:17:54.399704+0000 mgr.smithi153.asomxr (mgr.14195) 247 : cluster 0 pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:56.282 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:17:56.314 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:55 smithi153 bash[14340]: cluster 2024-03-12T02:17:54.399704+0000 mgr.smithi153.asomxr (mgr.14195) 247 : cluster 0 pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:58.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:57 smithi163 bash[22347]: cluster 2024-03-12T02:17:56.400152+0000 mgr.smithi153.asomxr (mgr.14195) 248 : cluster 0 pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:58.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:57 smithi163 bash[22347]: audit 2024-03-12T02:17:57.001438+0000 mon.smithi153 (mon.0) 756 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:58.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:57 smithi163 bash[22347]: audit 2024-03-12T02:17:57.009463+0000 mon.smithi153 (mon.0) 757 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:58.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:57 smithi163 bash[22347]: audit 2024-03-12T02:17:57.550428+0000 mon.smithi153 (mon.0) 758 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:58.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:57 smithi163 bash[22347]: audit 2024-03-12T02:17:57.557983+0000 mon.smithi153 (mon.0) 759 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:58.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:57 smithi163 bash[22347]: audit 2024-03-12T02:17:57.559567+0000 mon.smithi153 (mon.0) 760 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:17:58.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:57 smithi163 bash[22347]: audit 2024-03-12T02:17:57.560578+0000 mon.smithi153 (mon.0) 761 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:17:58.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:57 smithi163 bash[22347]: audit 2024-03-12T02:17:57.566933+0000 mon.smithi153 (mon.0) 762 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:58.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:57 smithi163 bash[22347]: audit 2024-03-12T02:17:57.570450+0000 mon.smithi153 (mon.0) 763 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:17:58.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:57 smithi153 bash[14340]: cluster 2024-03-12T02:17:56.400152+0000 mgr.smithi153.asomxr (mgr.14195) 248 : cluster 0 pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:17:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:57 smithi153 bash[14340]: audit 2024-03-12T02:17:57.001438+0000 mon.smithi153 (mon.0) 756 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:57 smithi153 bash[14340]: audit 2024-03-12T02:17:57.009463+0000 mon.smithi153 (mon.0) 757 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:57 smithi153 bash[14340]: audit 2024-03-12T02:17:57.550428+0000 mon.smithi153 (mon.0) 758 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:57 smithi153 bash[14340]: audit 2024-03-12T02:17:57.557983+0000 mon.smithi153 (mon.0) 759 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:57 smithi153 bash[14340]: audit 2024-03-12T02:17:57.559567+0000 mon.smithi153 (mon.0) 760 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:17:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:57 smithi153 bash[14340]: audit 2024-03-12T02:17:57.560578+0000 mon.smithi153 (mon.0) 761 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:17:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:57 smithi153 bash[14340]: audit 2024-03-12T02:17:57.566933+0000 mon.smithi153 (mon.0) 762 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:57 smithi153 bash[14340]: audit 2024-03-12T02:17:57.570450+0000 mon.smithi153 (mon.0) 763 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:17:59.224 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:58 smithi163 bash[22347]: cephadm 2024-03-12T02:17:57.578906+0000 mgr.smithi153.asomxr (mgr.14195) 249 : cephadm 1 Deploying daemon jaeger-agent.smithi163 on smithi163 2024-03-12T02:17:59.224 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:58 smithi163 bash[22347]: audit 2024-03-12T02:17:58.606999+0000 mon.smithi153 (mon.0) 764 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:59.224 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:58 smithi163 bash[22347]: audit 2024-03-12T02:17:58.615742+0000 mon.smithi153 (mon.0) 765 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:59.224 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:58 smithi163 bash[22347]: audit 2024-03-12T02:17:58.651677+0000 mon.smithi153 (mon.0) 766 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:59.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:58 smithi153 bash[14340]: cephadm 2024-03-12T02:17:57.578906+0000 mgr.smithi153.asomxr (mgr.14195) 249 : cephadm 1 Deploying daemon jaeger-agent.smithi163 on smithi163 2024-03-12T02:17:59.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:58 smithi153 bash[14340]: audit 2024-03-12T02:17:58.606999+0000 mon.smithi153 (mon.0) 764 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:59.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:58 smithi153 bash[14340]: audit 2024-03-12T02:17:58.615742+0000 mon.smithi153 (mon.0) 765 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:59.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:58 smithi153 bash[14340]: audit 2024-03-12T02:17:58.651677+0000 mon.smithi153 (mon.0) 766 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:17:59.476 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:17:59.476 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:17:57.541556Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:17:56.990363Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:17:56.990635Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "ports": [9200], "running": 0, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:17:57.541654Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "ports": [16686], "running": 0, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:17:56.991005Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:17:56.991195Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:17:56.990824Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:17:56.991458Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:17:57.541752Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:18:00.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:59 smithi153 bash[14340]: cluster 2024-03-12T02:17:58.400804+0000 mgr.smithi153.asomxr (mgr.14195) 250 : cluster 0 pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:00.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:59 smithi153 bash[14340]: cephadm 2024-03-12T02:17:58.653153+0000 mgr.smithi153.asomxr (mgr.14195) 251 : cephadm 1 Deploying daemon jaeger-agent.smithi153 on smithi153 2024-03-12T02:18:00.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:59 smithi153 bash[14340]: audit 2024-03-12T02:17:59.628548+0000 mon.smithi153 (mon.0) 767 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:00.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:59 smithi153 bash[14340]: audit 2024-03-12T02:17:59.636197+0000 mon.smithi153 (mon.0) 768 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:00.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:59 smithi153 bash[14340]: audit 2024-03-12T02:17:59.679224+0000 mon.smithi153 (mon.0) 769 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:00.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:17:59 smithi153 bash[14340]: audit 2024-03-12T02:17:59.684400+0000 mon.smithi153 (mon.0) 770 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:00.201 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:59 smithi163 bash[22347]: cluster 2024-03-12T02:17:58.400804+0000 mgr.smithi153.asomxr (mgr.14195) 250 : cluster 0 pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:00.201 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:59 smithi163 bash[22347]: cephadm 2024-03-12T02:17:58.653153+0000 mgr.smithi153.asomxr (mgr.14195) 251 : cephadm 1 Deploying daemon jaeger-agent.smithi153 on smithi153 2024-03-12T02:18:00.201 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:59 smithi163 bash[22347]: audit 2024-03-12T02:17:59.628548+0000 mon.smithi153 (mon.0) 767 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:00.201 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:59 smithi163 bash[22347]: audit 2024-03-12T02:17:59.636197+0000 mon.smithi153 (mon.0) 768 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:00.202 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:59 smithi163 bash[22347]: audit 2024-03-12T02:17:59.679224+0000 mon.smithi153 (mon.0) 769 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:00.202 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:17:59 smithi163 bash[22347]: audit 2024-03-12T02:17:59.684400+0000 mon.smithi153 (mon.0) 770 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:00.414 INFO:tasks.cephadm:elasticsearch has 0/1 2024-03-12T02:18:01.061 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:00 smithi163 bash[22347]: audit 2024-03-12T02:17:59.462525+0000 mgr.smithi153.asomxr (mgr.14195) 252 : audit 0 from='client.14502 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:01.062 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:00 smithi163 bash[22347]: cephadm 2024-03-12T02:17:59.687723+0000 mgr.smithi153.asomxr (mgr.14195) 253 : cephadm 1 Deploying daemon elasticsearch.smithi163 on smithi163 2024-03-12T02:18:01.062 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:00 smithi163 bash[22347]: audit 2024-03-12T02:18:00.378980+0000 mon.smithi153 (mon.0) 771 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:18:01.062 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:00 smithi163 bash[22347]: audit 2024-03-12T02:18:00.395158+0000 mon.smithi153 (mon.0) 772 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:01.062 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:00 smithi163 bash[22347]: audit 2024-03-12T02:18:00.759994+0000 mon.smithi153 (mon.0) 773 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:01.062 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:00 smithi163 bash[22347]: audit 2024-03-12T02:18:00.768570+0000 mon.smithi153 (mon.0) 774 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:01.062 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:00 smithi163 bash[22347]: audit 2024-03-12T02:18:00.811749+0000 mon.smithi153 (mon.0) 775 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:01.062 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:00 smithi163 bash[22347]: audit 2024-03-12T02:18:00.818167+0000 mon.smithi153 (mon.0) 776 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:01.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:00 smithi153 bash[14340]: audit 2024-03-12T02:17:59.462525+0000 mgr.smithi153.asomxr (mgr.14195) 252 : audit 0 from='client.14502 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:01.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:00 smithi153 bash[14340]: cephadm 2024-03-12T02:17:59.687723+0000 mgr.smithi153.asomxr (mgr.14195) 253 : cephadm 1 Deploying daemon elasticsearch.smithi163 on smithi163 2024-03-12T02:18:01.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:00 smithi153 bash[14340]: audit 2024-03-12T02:18:00.378980+0000 mon.smithi153 (mon.0) 771 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:18:01.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:00 smithi153 bash[14340]: audit 2024-03-12T02:18:00.395158+0000 mon.smithi153 (mon.0) 772 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:01.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:00 smithi153 bash[14340]: audit 2024-03-12T02:18:00.759994+0000 mon.smithi153 (mon.0) 773 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:01.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:00 smithi153 bash[14340]: audit 2024-03-12T02:18:00.768570+0000 mon.smithi153 (mon.0) 774 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:01.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:00 smithi153 bash[14340]: audit 2024-03-12T02:18:00.811749+0000 mon.smithi153 (mon.0) 775 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:01.239 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:00 smithi153 bash[14340]: audit 2024-03-12T02:18:00.818167+0000 mon.smithi153 (mon.0) 776 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:01.416 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:18:02.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:01 smithi153 bash[14340]: cluster 2024-03-12T02:18:00.401121+0000 mgr.smithi153.asomxr (mgr.14195) 254 : cluster 0 pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:02.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:01 smithi153 bash[14340]: cephadm 2024-03-12T02:18:00.820205+0000 mgr.smithi153.asomxr (mgr.14195) 255 : cephadm 1 Deploying daemon jaeger-collector.smithi153 on smithi153 2024-03-12T02:18:02.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:01 smithi153 bash[14340]: audit 2024-03-12T02:18:01.793593+0000 mon.smithi153 (mon.0) 777 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:02.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:01 smithi153 bash[14340]: audit 2024-03-12T02:18:01.802655+0000 mon.smithi153 (mon.0) 778 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:02.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:01 smithi153 bash[14340]: audit 2024-03-12T02:18:01.842514+0000 mon.smithi153 (mon.0) 779 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:02.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:01 smithi153 bash[14340]: audit 2024-03-12T02:18:01.847553+0000 mon.smithi153 (mon.0) 780 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:02.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:01 smithi163 bash[22347]: cluster 2024-03-12T02:18:00.401121+0000 mgr.smithi153.asomxr (mgr.14195) 254 : cluster 0 pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:02.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:01 smithi163 bash[22347]: cephadm 2024-03-12T02:18:00.820205+0000 mgr.smithi153.asomxr (mgr.14195) 255 : cephadm 1 Deploying daemon jaeger-collector.smithi153 on smithi153 2024-03-12T02:18:02.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:01 smithi163 bash[22347]: audit 2024-03-12T02:18:01.793593+0000 mon.smithi153 (mon.0) 777 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:02.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:01 smithi163 bash[22347]: audit 2024-03-12T02:18:01.802655+0000 mon.smithi153 (mon.0) 778 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:02.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:01 smithi163 bash[22347]: audit 2024-03-12T02:18:01.842514+0000 mon.smithi153 (mon.0) 779 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:02.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:01 smithi163 bash[22347]: audit 2024-03-12T02:18:01.847553+0000 mon.smithi153 (mon.0) 780 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:02.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:02 smithi153 bash[14340]: cephadm 2024-03-12T02:18:01.849630+0000 mgr.smithi153.asomxr (mgr.14195) 256 : cephadm 1 Deploying daemon jaeger-query.smithi163 on smithi163 2024-03-12T02:18:02.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:02 smithi153 bash[14340]: audit 2024-03-12T02:18:02.908172+0000 mon.smithi153 (mon.0) 781 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:02.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:02 smithi153 bash[14340]: audit 2024-03-12T02:18:02.914951+0000 mon.smithi153 (mon.0) 782 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:03.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:02 smithi163 bash[22347]: cephadm 2024-03-12T02:18:01.849630+0000 mgr.smithi153.asomxr (mgr.14195) 256 : cephadm 1 Deploying daemon jaeger-query.smithi163 on smithi163 2024-03-12T02:18:03.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:02 smithi163 bash[22347]: audit 2024-03-12T02:18:02.908172+0000 mon.smithi153 (mon.0) 781 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:03.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:02 smithi163 bash[22347]: audit 2024-03-12T02:18:02.914951+0000 mon.smithi153 (mon.0) 782 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:04.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:03 smithi163 bash[22347]: cluster 2024-03-12T02:18:02.401461+0000 mgr.smithi153.asomxr (mgr.14195) 257 : cluster 0 pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:04.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:03 smithi163 bash[22347]: audit 2024-03-12T02:18:02.972386+0000 mon.smithi153 (mon.0) 783 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:04.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:03 smithi163 bash[22347]: audit 2024-03-12T02:18:02.977236+0000 mon.smithi153 (mon.0) 784 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:04.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:03 smithi163 bash[22347]: audit 2024-03-12T02:18:03.042604+0000 mon.smithi153 (mon.0) 785 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:18:04.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:03 smithi153 bash[14340]: cluster 2024-03-12T02:18:02.401461+0000 mgr.smithi153.asomxr (mgr.14195) 257 : cluster 0 pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:04.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:03 smithi153 bash[14340]: audit 2024-03-12T02:18:02.972386+0000 mon.smithi153 (mon.0) 783 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:04.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:03 smithi153 bash[14340]: audit 2024-03-12T02:18:02.977236+0000 mon.smithi153 (mon.0) 784 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:04.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:03 smithi153 bash[14340]: audit 2024-03-12T02:18:03.042604+0000 mon.smithi153 (mon.0) 785 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:18:05.153 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:04 smithi153 bash[14340]: cluster 2024-03-12T02:18:04.401988+0000 mgr.smithi153.asomxr (mgr.14195) 258 : cluster 0 pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:05.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:04 smithi163 bash[22347]: cluster 2024-03-12T02:18:04.401988+0000 mgr.smithi153.asomxr (mgr.14195) 258 : cluster 0 pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:06.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:06 smithi163 bash[22347]: audit 2024-03-12T02:18:05.404915+0000 mon.smithi153 (mon.0) 786 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:06.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:06 smithi153 bash[14340]: audit 2024-03-12T02:18:05.404915+0000 mon.smithi153 (mon.0) 786 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:06.870 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:18:07.722 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:07 smithi163 bash[22347]: cluster 2024-03-12T02:18:06.402467+0000 mgr.smithi153.asomxr (mgr.14195) 259 : cluster 0 pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:07.722 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:07 smithi153 bash[14340]: cluster 2024-03-12T02:18:06.402467+0000 mgr.smithi153.asomxr (mgr.14195) 259 : cluster 0 pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:10.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:09 smithi153 bash[14340]: cluster 2024-03-12T02:18:08.403085+0000 mgr.smithi153.asomxr (mgr.14195) 260 : cluster 0 pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:10.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:09 smithi153 bash[14340]: audit 2024-03-12T02:18:08.887744+0000 mon.smithi153 (mon.0) 787 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:10.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:09 smithi153 bash[14340]: audit 2024-03-12T02:18:08.894616+0000 mon.smithi153 (mon.0) 788 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:10.160 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:18:10.160 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:17:57.541556Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:17:57.541254Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:17:57.541357Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:18:08.877850Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:17:57.541654Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:18:08.877645Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:18:08.878043Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:17:57.541133Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:17:57.541015Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:17:57.541457Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:17:57.541851Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:17:57.541752Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:18:10.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:09 smithi163 bash[22347]: cluster 2024-03-12T02:18:08.403085+0000 mgr.smithi153.asomxr (mgr.14195) 260 : cluster 0 pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:10.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:09 smithi163 bash[22347]: audit 2024-03-12T02:18:08.887744+0000 mon.smithi153 (mon.0) 787 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:10.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:09 smithi163 bash[22347]: audit 2024-03-12T02:18:08.894616+0000 mon.smithi153 (mon.0) 788 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:10.958 INFO:tasks.cephadm:elasticsearch has 1/1 2024-03-12T02:18:10.959 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service... 2024-03-12T02:18:10.968 INFO:tasks.cephadm:Waiting for ceph service jaeger-collector to start (timeout 300)... 2024-03-12T02:18:10.969 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:18:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:11 smithi153 bash[14340]: audit 2024-03-12T02:18:10.150955+0000 mgr.smithi153.asomxr (mgr.14195) 261 : audit 0 from='client.14506 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:11 smithi153 bash[14340]: cluster 2024-03-12T02:18:10.403501+0000 mgr.smithi153.asomxr (mgr.14195) 262 : cluster 0 pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:11 smithi153 bash[14340]: audit 2024-03-12T02:18:10.793124+0000 mon.smithi153 (mon.0) 789 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:11 smithi153 bash[14340]: audit 2024-03-12T02:18:10.801743+0000 mon.smithi153 (mon.0) 790 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:11 smithi153 bash[14340]: audit 2024-03-12T02:18:10.803247+0000 mon.smithi153 (mon.0) 791 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:18:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:11 smithi153 bash[14340]: audit 2024-03-12T02:18:10.804531+0000 mon.smithi153 (mon.0) 792 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:18:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:11 smithi153 bash[14340]: cluster 2024-03-12T02:18:10.806831+0000 mgr.smithi153.asomxr (mgr.14195) 263 : cluster 0 pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:11 smithi153 bash[14340]: audit 2024-03-12T02:18:10.810587+0000 mon.smithi153 (mon.0) 793 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:11 smithi153 bash[14340]: audit 2024-03-12T02:18:10.816114+0000 mon.smithi153 (mon.0) 794 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:18:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:11 smithi153 bash[14340]: cluster 2024-03-12T02:18:10.886108+0000 mon.smithi153 (mon.0) 795 : cluster 3 Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-12T02:18:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:11 smithi153 bash[14340]: audit 2024-03-12T02:18:10.903089+0000 mon.smithi153 (mon.0) 796 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:18:12.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:11 smithi163 bash[22347]: audit 2024-03-12T02:18:10.150955+0000 mgr.smithi153.asomxr (mgr.14195) 261 : audit 0 from='client.14506 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:12.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:11 smithi163 bash[22347]: cluster 2024-03-12T02:18:10.403501+0000 mgr.smithi153.asomxr (mgr.14195) 262 : cluster 0 pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:12.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:11 smithi163 bash[22347]: audit 2024-03-12T02:18:10.793124+0000 mon.smithi153 (mon.0) 789 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:12.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:11 smithi163 bash[22347]: audit 2024-03-12T02:18:10.801743+0000 mon.smithi153 (mon.0) 790 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:12.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:11 smithi163 bash[22347]: audit 2024-03-12T02:18:10.803247+0000 mon.smithi153 (mon.0) 791 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:18:12.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:11 smithi163 bash[22347]: audit 2024-03-12T02:18:10.804531+0000 mon.smithi153 (mon.0) 792 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:18:12.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:11 smithi163 bash[22347]: cluster 2024-03-12T02:18:10.806831+0000 mgr.smithi153.asomxr (mgr.14195) 263 : cluster 0 pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:12.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:11 smithi163 bash[22347]: audit 2024-03-12T02:18:10.810587+0000 mon.smithi153 (mon.0) 793 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:12.193 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:11 smithi163 bash[22347]: audit 2024-03-12T02:18:10.816114+0000 mon.smithi153 (mon.0) 794 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:18:12.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:11 smithi163 bash[22347]: cluster 2024-03-12T02:18:10.886108+0000 mon.smithi153 (mon.0) 795 : cluster 3 Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-12T02:18:12.194 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:11 smithi163 bash[22347]: audit 2024-03-12T02:18:10.903089+0000 mon.smithi153 (mon.0) 796 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:18:13.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:13 smithi153 bash[14340]: cluster 2024-03-12T02:18:12.807352+0000 mgr.smithi153.asomxr (mgr.14195) 264 : cluster 0 pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:13.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:13 smithi163 bash[22347]: cluster 2024-03-12T02:18:12.807352+0000 mgr.smithi153.asomxr (mgr.14195) 264 : cluster 0 pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:16.051 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:18:16.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:15 smithi163 bash[22347]: cluster 2024-03-12T02:18:14.808105+0000 mgr.smithi153.asomxr (mgr.14195) 265 : cluster 0 pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:16.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:15 smithi163 bash[22347]: audit 2024-03-12T02:18:15.385139+0000 mon.smithi153 (mon.0) 797 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:16.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:15 smithi163 bash[22347]: audit 2024-03-12T02:18:15.386059+0000 mon.smithi153 (mon.0) 798 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:18:16.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:15 smithi153 bash[14340]: cluster 2024-03-12T02:18:14.808105+0000 mgr.smithi153.asomxr (mgr.14195) 265 : cluster 0 pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:15 smithi153 bash[14340]: audit 2024-03-12T02:18:15.385139+0000 mon.smithi153 (mon.0) 797 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:15 smithi153 bash[14340]: audit 2024-03-12T02:18:15.386059+0000 mon.smithi153 (mon.0) 798 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:18:17.858 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:17 smithi153 bash[14340]: audit 2024-03-12T02:18:16.596805+0000 mon.smithi153 (mon.0) 799 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:17.858 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:17 smithi153 bash[14340]: audit 2024-03-12T02:18:16.603481+0000 mon.smithi153 (mon.0) 800 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:17.858 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:17 smithi153 bash[14340]: cluster 2024-03-12T02:18:16.808689+0000 mgr.smithi153.asomxr (mgr.14195) 266 : cluster 0 pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:17.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:17 smithi163 bash[22347]: audit 2024-03-12T02:18:16.596805+0000 mon.smithi153 (mon.0) 799 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:17.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:17 smithi163 bash[22347]: audit 2024-03-12T02:18:16.603481+0000 mon.smithi153 (mon.0) 800 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:17.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:17 smithi163 bash[22347]: cluster 2024-03-12T02:18:16.808689+0000 mgr.smithi153.asomxr (mgr.14195) 266 : cluster 0 pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:18.980 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:18 smithi153 bash[14340]: audit 2024-03-12T02:18:17.705229+0000 mon.smithi153 (mon.0) 801 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:18.980 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:18 smithi153 bash[14340]: audit 2024-03-12T02:18:17.711330+0000 mon.smithi153 (mon.0) 802 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:18.980 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:18 smithi153 bash[14340]: audit 2024-03-12T02:18:17.712702+0000 mon.smithi153 (mon.0) 803 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:18:18.980 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:18 smithi153 bash[14340]: audit 2024-03-12T02:18:17.713909+0000 mon.smithi153 (mon.0) 804 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:18:18.980 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:18 smithi153 bash[14340]: cluster 2024-03-12T02:18:17.716031+0000 mgr.smithi153.asomxr (mgr.14195) 267 : cluster 0 pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:18.980 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:18 smithi153 bash[14340]: cluster 2024-03-12T02:18:17.716255+0000 mgr.smithi153.asomxr (mgr.14195) 268 : cluster 0 pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:18.981 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:18 smithi153 bash[14340]: audit 2024-03-12T02:18:17.721510+0000 mon.smithi153 (mon.0) 805 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:18.981 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:18 smithi153 bash[14340]: audit 2024-03-12T02:18:17.726175+0000 mon.smithi153 (mon.0) 806 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:18:18.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:18 smithi163 bash[22347]: audit 2024-03-12T02:18:17.705229+0000 mon.smithi153 (mon.0) 801 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:18.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:18 smithi163 bash[22347]: audit 2024-03-12T02:18:17.711330+0000 mon.smithi153 (mon.0) 802 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:18.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:18 smithi163 bash[22347]: audit 2024-03-12T02:18:17.712702+0000 mon.smithi153 (mon.0) 803 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:18:18.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:18 smithi163 bash[22347]: audit 2024-03-12T02:18:17.713909+0000 mon.smithi153 (mon.0) 804 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:18:18.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:18 smithi163 bash[22347]: cluster 2024-03-12T02:18:17.716031+0000 mgr.smithi153.asomxr (mgr.14195) 267 : cluster 0 pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:18.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:18 smithi163 bash[22347]: cluster 2024-03-12T02:18:17.716255+0000 mgr.smithi153.asomxr (mgr.14195) 268 : cluster 0 pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:18.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:18 smithi163 bash[22347]: audit 2024-03-12T02:18:17.721510+0000 mon.smithi153 (mon.0) 805 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:18:18.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:18 smithi163 bash[22347]: audit 2024-03-12T02:18:17.726175+0000 mon.smithi153 (mon.0) 806 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:18:19.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:19 smithi163 bash[22347]: cluster 2024-03-12T02:18:18.710334+0000 mon.smithi153 (mon.0) 807 : cluster 3 Health check update: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-12T02:18:19.991 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:19 smithi153 bash[14340]: cluster 2024-03-12T02:18:18.710334+0000 mon.smithi153 (mon.0) 807 : cluster 3 Health check update: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-12T02:18:20.404 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:18:20.404 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:18:17.694378Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:18:16.587517Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:18:16.587740Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:18:16.589312Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:18:17.694658Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:18:16.589161Z", "ports": [6799], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:18:17.696329Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:18:16.589424Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:18:16.588122Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:18:16.588310Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:18:16.587932Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:18:16.588495Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:18:17.694905Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:18:21.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:20 smithi153 bash[14340]: cluster 2024-03-12T02:18:19.716591+0000 mgr.smithi153.asomxr (mgr.14195) 269 : cluster 0 pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:21.201 INFO:tasks.cephadm:jaeger-collector has 0/1 2024-03-12T02:18:21.279 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:20 smithi163 bash[22347]: cluster 2024-03-12T02:18:19.716591+0000 mgr.smithi153.asomxr (mgr.14195) 269 : cluster 0 pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:21.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:21 smithi163 bash[22347]: audit 2024-03-12T02:18:20.390008+0000 mgr.smithi153.asomxr (mgr.14195) 270 : audit 0 from='client.14510 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:22.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:21 smithi153 bash[14340]: audit 2024-03-12T02:18:20.390008+0000 mgr.smithi153.asomxr (mgr.14195) 270 : audit 0 from='client.14510 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:22.202 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:18:22.857 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:22 smithi163 bash[22347]: cluster 2024-03-12T02:18:21.717236+0000 mgr.smithi153.asomxr (mgr.14195) 271 : cluster 0 pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:23.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:22 smithi153 bash[14340]: cluster 2024-03-12T02:18:21.717236+0000 mgr.smithi153.asomxr (mgr.14195) 271 : cluster 0 pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:24.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:24 smithi163 bash[22347]: cluster 2024-03-12T02:18:23.717741+0000 mgr.smithi153.asomxr (mgr.14195) 272 : cluster 0 pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:25.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:24 smithi153 bash[14340]: cluster 2024-03-12T02:18:23.717741+0000 mgr.smithi153.asomxr (mgr.14195) 272 : cluster 0 pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:26.947 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:18:27.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:26 smithi153 bash[14340]: cluster 2024-03-12T02:18:25.718228+0000 mgr.smithi153.asomxr (mgr.14195) 273 : cluster 0 pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:27.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:26 smithi163 bash[22347]: cluster 2024-03-12T02:18:25.718228+0000 mgr.smithi153.asomxr (mgr.14195) 273 : cluster 0 pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:28.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:28 smithi163 bash[22347]: cluster 2024-03-12T02:18:27.718774+0000 mgr.smithi153.asomxr (mgr.14195) 274 : cluster 0 pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:29.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:28 smithi153 bash[14340]: cluster 2024-03-12T02:18:27.718774+0000 mgr.smithi153.asomxr (mgr.14195) 274 : cluster 0 pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:30.865 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:30 smithi153 bash[14340]: cluster 2024-03-12T02:18:29.719458+0000 mgr.smithi153.asomxr (mgr.14195) 275 : cluster 0 pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:30.865 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:30 smithi153 bash[14340]: audit 2024-03-12T02:18:30.380864+0000 mon.smithi153 (mon.0) 808 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:18:31.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:30 smithi163 bash[22347]: cluster 2024-03-12T02:18:29.719458+0000 mgr.smithi153.asomxr (mgr.14195) 275 : cluster 0 pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:31.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:30 smithi163 bash[22347]: audit 2024-03-12T02:18:30.380864+0000 mon.smithi153 (mon.0) 808 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:18:31.325 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:18:31.325 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:18:17.694378Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:18:16.587517Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:18:16.587740Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:18:16.589312Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:18:17.694658Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:18:16.589161Z", "ports": [6799], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:18:17.696329Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:18:16.589424Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:18:16.588122Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:18:16.588310Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:18:16.587932Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:18:16.588495Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:18:17.694905Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:18:32.393 INFO:tasks.cephadm:jaeger-collector has 0/1 2024-03-12T02:18:32.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:32 smithi163 bash[22347]: audit 2024-03-12T02:18:31.311676+0000 mgr.smithi153.asomxr (mgr.14195) 276 : audit 0 from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:32.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:32 smithi163 bash[22347]: cluster 2024-03-12T02:18:31.720108+0000 mgr.smithi153.asomxr (mgr.14195) 277 : cluster 0 pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:33.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:32 smithi153 bash[14340]: audit 2024-03-12T02:18:31.311676+0000 mgr.smithi153.asomxr (mgr.14195) 276 : audit 0 from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:33.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:32 smithi153 bash[14340]: cluster 2024-03-12T02:18:31.720108+0000 mgr.smithi153.asomxr (mgr.14195) 277 : cluster 0 pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:33.394 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:18:34.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:34 smithi163 bash[22347]: cluster 2024-03-12T02:18:33.720720+0000 mgr.smithi153.asomxr (mgr.14195) 278 : cluster 0 pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:35.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:34 smithi153 bash[14340]: cluster 2024-03-12T02:18:33.720720+0000 mgr.smithi153.asomxr (mgr.14195) 278 : cluster 0 pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:36.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:36 smithi163 bash[22347]: cluster 2024-03-12T02:18:35.721397+0000 mgr.smithi153.asomxr (mgr.14195) 279 : cluster 0 pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:37.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:36 smithi153 bash[14340]: cluster 2024-03-12T02:18:35.721397+0000 mgr.smithi153.asomxr (mgr.14195) 279 : cluster 0 pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:38.142 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:18:38.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:38 smithi163 bash[22347]: cluster 2024-03-12T02:18:37.721963+0000 mgr.smithi153.asomxr (mgr.14195) 280 : cluster 0 pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:39.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:38 smithi153 bash[14340]: cluster 2024-03-12T02:18:37.721963+0000 mgr.smithi153.asomxr (mgr.14195) 280 : cluster 0 pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:41.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:40 smithi153 bash[14340]: cluster 2024-03-12T02:18:39.722553+0000 mgr.smithi153.asomxr (mgr.14195) 281 : cluster 0 pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:41.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:40 smithi163 bash[22347]: cluster 2024-03-12T02:18:39.722553+0000 mgr.smithi153.asomxr (mgr.14195) 281 : cluster 0 pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:41.322 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:18:41.322 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:18:17.694378Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:18:16.587517Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:18:16.587740Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:18:16.589312Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:18:17.694658Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:18:16.589161Z", "ports": [6799], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:18:17.696329Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:18:16.589424Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:18:16.588122Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:18:16.588310Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:18:16.587932Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:18:16.588495Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:18:17.694905Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:18:42.362 INFO:tasks.cephadm:jaeger-collector has 0/1 2024-03-12T02:18:42.959 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:42 smithi153 bash[14340]: audit 2024-03-12T02:18:41.311274+0000 mgr.smithi153.asomxr (mgr.14195) 282 : audit 0 from='client.14518 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:42.959 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:42 smithi153 bash[14340]: cluster 2024-03-12T02:18:41.722960+0000 mgr.smithi153.asomxr (mgr.14195) 283 : cluster 0 pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:43.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:42 smithi163 bash[22347]: audit 2024-03-12T02:18:41.311274+0000 mgr.smithi153.asomxr (mgr.14195) 282 : audit 0 from='client.14518 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:43.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:42 smithi163 bash[22347]: cluster 2024-03-12T02:18:41.722960+0000 mgr.smithi153.asomxr (mgr.14195) 283 : cluster 0 pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:43.364 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:18:45.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:44 smithi153 bash[14340]: cluster 2024-03-12T02:18:43.723444+0000 mgr.smithi153.asomxr (mgr.14195) 284 : cluster 0 pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:45.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:44 smithi163 bash[22347]: cluster 2024-03-12T02:18:43.723444+0000 mgr.smithi153.asomxr (mgr.14195) 284 : cluster 0 pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:46.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:45 smithi153 bash[14340]: audit 2024-03-12T02:18:45.381525+0000 mon.smithi153 (mon.0) 809 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:18:46.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:45 smithi163 bash[22347]: audit 2024-03-12T02:18:45.381525+0000 mon.smithi153 (mon.0) 809 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:18:47.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:46 smithi153 bash[14340]: cluster 2024-03-12T02:18:45.724059+0000 mgr.smithi153.asomxr (mgr.14195) 285 : cluster 0 pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:47.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:46 smithi163 bash[22347]: cluster 2024-03-12T02:18:45.724059+0000 mgr.smithi153.asomxr (mgr.14195) 285 : cluster 0 pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:48.111 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:18:48.900 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:48 smithi163 bash[22347]: cluster 2024-03-12T02:18:47.724614+0000 mgr.smithi153.asomxr (mgr.14195) 286 : cluster 0 pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:49.015 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:48 smithi153 bash[14340]: cluster 2024-03-12T02:18:47.724614+0000 mgr.smithi153.asomxr (mgr.14195) 286 : cluster 0 pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:51.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:50 smithi153 bash[14340]: cluster 2024-03-12T02:18:49.725241+0000 mgr.smithi153.asomxr (mgr.14195) 287 : cluster 0 pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:51.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:50 smithi163 bash[22347]: cluster 2024-03-12T02:18:49.725241+0000 mgr.smithi153.asomxr (mgr.14195) 287 : cluster 0 pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:51.254 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:18:51.254 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:18:17.694378Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:18:16.587517Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:18:16.587740Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:18:16.589312Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:18:17.694658Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:18:16.589161Z", "ports": [6799], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:18:17.696329Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:18:16.589424Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:18:16.588122Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:18:16.588310Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:18:16.587932Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:18:16.588495Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:18:17.694905Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:18:52.110 INFO:tasks.cephadm:jaeger-collector has 0/1 2024-03-12T02:18:53.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:52 smithi153 bash[14340]: audit 2024-03-12T02:18:51.238837+0000 mgr.smithi153.asomxr (mgr.14195) 288 : audit 0 from='client.14522 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:53.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:52 smithi153 bash[14340]: cluster 2024-03-12T02:18:51.725688+0000 mgr.smithi153.asomxr (mgr.14195) 289 : cluster 0 pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:53.110 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:18:53.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:52 smithi163 bash[22347]: audit 2024-03-12T02:18:51.238837+0000 mgr.smithi153.asomxr (mgr.14195) 288 : audit 0 from='client.14522 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:18:53.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:52 smithi163 bash[22347]: cluster 2024-03-12T02:18:51.725688+0000 mgr.smithi153.asomxr (mgr.14195) 289 : cluster 0 pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:55.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:54 smithi153 bash[14340]: cluster 2024-03-12T02:18:53.726263+0000 mgr.smithi153.asomxr (mgr.14195) 290 : cluster 0 pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:55.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:54 smithi163 bash[22347]: cluster 2024-03-12T02:18:53.726263+0000 mgr.smithi153.asomxr (mgr.14195) 290 : cluster 0 pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:57.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:56 smithi153 bash[14340]: cluster 2024-03-12T02:18:55.726859+0000 mgr.smithi153.asomxr (mgr.14195) 291 : cluster 0 pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:57.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:56 smithi163 bash[22347]: cluster 2024-03-12T02:18:55.726859+0000 mgr.smithi153.asomxr (mgr.14195) 291 : cluster 0 pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:57.861 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:18:58.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:18:58 smithi163 bash[22347]: cluster 2024-03-12T02:18:57.727452+0000 mgr.smithi153.asomxr (mgr.14195) 292 : cluster 0 pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:18:59.045 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:18:58 smithi153 bash[14340]: cluster 2024-03-12T02:18:57.727452+0000 mgr.smithi153.asomxr (mgr.14195) 292 : cluster 0 pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:00.907 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:19:00.907 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:18:17.694378Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:18:16.587517Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:18:16.587740Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:18:16.589312Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:18:17.694658Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:18:16.589161Z", "ports": [6799], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:18:17.696329Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:18:16.589424Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:18:16.588122Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:18:16.588310Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:18:16.587932Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:18:16.588495Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:18:17.694905Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:19:01.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:00 smithi153 bash[14340]: cluster 2024-03-12T02:18:59.728098+0000 mgr.smithi153.asomxr (mgr.14195) 293 : cluster 0 pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:01.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:00 smithi153 bash[14340]: audit 2024-03-12T02:19:00.382013+0000 mon.smithi153 (mon.0) 810 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:19:01.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:00 smithi163 bash[22347]: cluster 2024-03-12T02:18:59.728098+0000 mgr.smithi153.asomxr (mgr.14195) 293 : cluster 0 pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:01.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:00 smithi163 bash[22347]: audit 2024-03-12T02:19:00.382013+0000 mon.smithi153 (mon.0) 810 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:19:01.658 INFO:tasks.cephadm:jaeger-collector has 0/1 2024-03-12T02:19:02.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:01 smithi153 bash[14340]: audit 2024-03-12T02:19:00.893823+0000 mgr.smithi153.asomxr (mgr.14195) 294 : audit 0 from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:02.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:01 smithi163 bash[22347]: audit 2024-03-12T02:19:00.893823+0000 mgr.smithi153.asomxr (mgr.14195) 294 : audit 0 from='client.14526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:02.659 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:19:02.959 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:02 smithi153 bash[14340]: cluster 2024-03-12T02:19:01.728745+0000 mgr.smithi153.asomxr (mgr.14195) 295 : cluster 0 pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:03.170 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:02 smithi163 bash[22347]: cluster 2024-03-12T02:19:01.728745+0000 mgr.smithi153.asomxr (mgr.14195) 295 : cluster 0 pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:05.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:04 smithi153 bash[14340]: cluster 2024-03-12T02:19:03.729368+0000 mgr.smithi153.asomxr (mgr.14195) 296 : cluster 0 pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:05.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:04 smithi163 bash[22347]: cluster 2024-03-12T02:19:03.729368+0000 mgr.smithi153.asomxr (mgr.14195) 296 : cluster 0 pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:07.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:06 smithi153 bash[14340]: cluster 2024-03-12T02:19:05.729986+0000 mgr.smithi153.asomxr (mgr.14195) 297 : cluster 0 pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:07.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:06 smithi163 bash[22347]: cluster 2024-03-12T02:19:05.729986+0000 mgr.smithi153.asomxr (mgr.14195) 297 : cluster 0 pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:07.408 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:19:08.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:08 smithi163 bash[22347]: cluster 2024-03-12T02:19:07.730616+0000 mgr.smithi153.asomxr (mgr.14195) 298 : cluster 0 pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:09.147 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:08 smithi153 bash[14340]: cluster 2024-03-12T02:19:07.730616+0000 mgr.smithi153.asomxr (mgr.14195) 298 : cluster 0 pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:10.997 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:19:10.997 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:18:17.694378Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:18:16.587517Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:18:16.587740Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:18:16.589312Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:18:17.694658Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:18:16.589161Z", "ports": [6799], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:18:17.696329Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:18:16.589424Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:18:16.588122Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:18:16.588310Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:18:16.587932Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:18:16.588495Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:18:17.694905Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:19:11.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:10 smithi153 bash[14340]: cluster 2024-03-12T02:19:09.731270+0000 mgr.smithi153.asomxr (mgr.14195) 299 : cluster 0 pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:11.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:10 smithi163 bash[22347]: cluster 2024-03-12T02:19:09.731270+0000 mgr.smithi153.asomxr (mgr.14195) 299 : cluster 0 pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:11.891 INFO:tasks.cephadm:jaeger-collector has 0/1 2024-03-12T02:19:12.892 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:19:13.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:12 smithi153 bash[14340]: audit 2024-03-12T02:19:10.982871+0000 mgr.smithi153.asomxr (mgr.14195) 300 : audit 0 from='client.14530 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:13.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:12 smithi153 bash[14340]: cluster 2024-03-12T02:19:11.731886+0000 mgr.smithi153.asomxr (mgr.14195) 301 : cluster 0 pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:13.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:12 smithi163 bash[22347]: audit 2024-03-12T02:19:10.982871+0000 mgr.smithi153.asomxr (mgr.14195) 300 : audit 0 from='client.14530 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:13.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:12 smithi163 bash[22347]: cluster 2024-03-12T02:19:11.731886+0000 mgr.smithi153.asomxr (mgr.14195) 301 : cluster 0 pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:15.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:14 smithi153 bash[14340]: cluster 2024-03-12T02:19:13.732490+0000 mgr.smithi153.asomxr (mgr.14195) 302 : cluster 0 pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:15.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:14 smithi163 bash[22347]: cluster 2024-03-12T02:19:13.732490+0000 mgr.smithi153.asomxr (mgr.14195) 302 : cluster 0 pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:16.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:15 smithi153 bash[14340]: audit 2024-03-12T02:19:15.381961+0000 mon.smithi153 (mon.0) 811 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:19:16.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:15 smithi163 bash[22347]: audit 2024-03-12T02:19:15.381961+0000 mon.smithi153 (mon.0) 811 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:19:17.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:16 smithi153 bash[14340]: cluster 2024-03-12T02:19:15.733078+0000 mgr.smithi153.asomxr (mgr.14195) 303 : cluster 0 pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:17.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:16 smithi163 bash[22347]: cluster 2024-03-12T02:19:15.733078+0000 mgr.smithi153.asomxr (mgr.14195) 303 : cluster 0 pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:17.648 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:19:18.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:18 smithi163 bash[22347]: cluster 2024-03-12T02:19:17.733702+0000 mgr.smithi153.asomxr (mgr.14195) 304 : cluster 0 pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:18.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:18 smithi163 bash[22347]: audit 2024-03-12T02:19:17.813656+0000 mon.smithi153 (mon.0) 812 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:19:18.942 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:18 smithi153 bash[14340]: cluster 2024-03-12T02:19:17.733702+0000 mgr.smithi153.asomxr (mgr.14195) 304 : cluster 0 pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:18.943 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:18 smithi153 bash[14340]: audit 2024-03-12T02:19:17.813656+0000 mon.smithi153 (mon.0) 812 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:18:17.694378Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:18:16.587517Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:18:16.587740Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:18:16.589312Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:18:17.694658Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:18:16.589161Z", "ports": [6799], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:18:17.696329Z", "ports": [14250], "running": 0, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:18:16.589424Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:18:16.588122Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:18:16.588310Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:18:16.587932Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:18:16.588495Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:18:17.694905Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:19:21.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:20 smithi153 bash[14340]: cluster 2024-03-12T02:19:19.734385+0000 mgr.smithi153.asomxr (mgr.14195) 305 : cluster 0 pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:21.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:20 smithi163 bash[22347]: cluster 2024-03-12T02:19:19.734385+0000 mgr.smithi153.asomxr (mgr.14195) 305 : cluster 0 pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:21.631 INFO:tasks.cephadm:jaeger-collector has 0/1 2024-03-12T02:19:22.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:21 smithi153 bash[14340]: audit 2024-03-12T02:19:20.863149+0000 mgr.smithi153.asomxr (mgr.14195) 306 : audit 0 from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:22.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:21 smithi163 bash[22347]: audit 2024-03-12T02:19:20.863149+0000 mgr.smithi153.asomxr (mgr.14195) 306 : audit 0 from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:22.632 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:19:22.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:22 smithi153 bash[14340]: cluster 2024-03-12T02:19:21.734976+0000 mgr.smithi153.asomxr (mgr.14195) 307 : cluster 0 pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:23.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:22 smithi163 bash[22347]: cluster 2024-03-12T02:19:21.734976+0000 mgr.smithi153.asomxr (mgr.14195) 307 : cluster 0 pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:24.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:24 smithi163 bash[22347]: audit 2024-03-12T02:19:23.665066+0000 mon.smithi153 (mon.0) 813 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:19:24.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:24 smithi163 bash[22347]: audit 2024-03-12T02:19:23.669615+0000 mon.smithi153 (mon.0) 814 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:19:24.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:24 smithi163 bash[22347]: audit 2024-03-12T02:19:23.677039+0000 mon.smithi153 (mon.0) 815 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:19:24.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:24 smithi163 bash[22347]: audit 2024-03-12T02:19:23.682846+0000 mon.smithi153 (mon.0) 816 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:19:24.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:24 smithi163 bash[22347]: cluster 2024-03-12T02:19:23.735660+0000 mgr.smithi153.asomxr (mgr.14195) 308 : cluster 0 pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:24.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:24 smithi163 bash[22347]: audit 2024-03-12T02:19:24.209082+0000 mon.smithi153 (mon.0) 817 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:19:24.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:24 smithi163 bash[22347]: audit 2024-03-12T02:19:24.210626+0000 mon.smithi153 (mon.0) 818 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:19:24.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:24 smithi163 bash[22347]: audit 2024-03-12T02:19:24.219900+0000 mon.smithi153 (mon.0) 819 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:19:24.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:24 smithi163 bash[22347]: audit 2024-03-12T02:19:24.224780+0000 mon.smithi153 (mon.0) 820 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:19:25.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:24 smithi153 bash[14340]: audit 2024-03-12T02:19:23.665066+0000 mon.smithi153 (mon.0) 813 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:19:25.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:24 smithi153 bash[14340]: audit 2024-03-12T02:19:23.669615+0000 mon.smithi153 (mon.0) 814 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:19:25.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:24 smithi153 bash[14340]: audit 2024-03-12T02:19:23.677039+0000 mon.smithi153 (mon.0) 815 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:19:25.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:24 smithi153 bash[14340]: audit 2024-03-12T02:19:23.682846+0000 mon.smithi153 (mon.0) 816 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:19:25.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:24 smithi153 bash[14340]: cluster 2024-03-12T02:19:23.735660+0000 mgr.smithi153.asomxr (mgr.14195) 308 : cluster 0 pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:25.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:24 smithi153 bash[14340]: audit 2024-03-12T02:19:24.209082+0000 mon.smithi153 (mon.0) 817 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:19:25.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:24 smithi153 bash[14340]: audit 2024-03-12T02:19:24.210626+0000 mon.smithi153 (mon.0) 818 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:19:25.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:24 smithi153 bash[14340]: audit 2024-03-12T02:19:24.219900+0000 mon.smithi153 (mon.0) 819 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:19:25.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:24 smithi153 bash[14340]: audit 2024-03-12T02:19:24.224780+0000 mon.smithi153 (mon.0) 820 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:19:25.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:25 smithi163 bash[22347]: cluster 2024-03-12T02:19:24.213119+0000 mgr.smithi153.asomxr (mgr.14195) 309 : cluster 0 pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:25.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:25 smithi163 bash[22347]: cluster 2024-03-12T02:19:24.213528+0000 mgr.smithi153.asomxr (mgr.14195) 310 : cluster 0 pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:25.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:25 smithi163 bash[22347]: cluster 2024-03-12T02:19:24.680204+0000 mon.smithi153 (mon.0) 821 : cluster 3 Health check update: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-12T02:19:26.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:25 smithi153 bash[14340]: cluster 2024-03-12T02:19:24.213119+0000 mgr.smithi153.asomxr (mgr.14195) 309 : cluster 0 pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:26.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:25 smithi153 bash[14340]: cluster 2024-03-12T02:19:24.213528+0000 mgr.smithi153.asomxr (mgr.14195) 310 : cluster 0 pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:26.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:25 smithi153 bash[14340]: cluster 2024-03-12T02:19:24.680204+0000 mon.smithi153 (mon.0) 821 : cluster 3 Health check update: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-03-12T02:19:27.386 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:19:27.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:27 smithi163 bash[22347]: cluster 2024-03-12T02:19:26.214169+0000 mgr.smithi153.asomxr (mgr.14195) 311 : cluster 0 pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:28.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:27 smithi153 bash[14340]: cluster 2024-03-12T02:19:26.214169+0000 mgr.smithi153.asomxr (mgr.14195) 311 : cluster 0 pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:29.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:29 smithi163 bash[22347]: cluster 2024-03-12T02:19:28.214657+0000 mgr.smithi153.asomxr (mgr.14195) 312 : cluster 0 pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:30.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:29 smithi153 bash[14340]: cluster 2024-03-12T02:19:28.214657+0000 mgr.smithi153.asomxr (mgr.14195) 312 : cluster 0 pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:30.643 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:19:30.644 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:19:23.648341Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:19:23.647604Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:19:23.647849Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:19:23.654432Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:19:23.648584Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:19:23.649985Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:19:23.650224Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:19:23.654656Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:19:23.647352Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:19:23.647074Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:19:23.648090Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:19:23.649064Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:19:23.648824Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:19:30.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:30 smithi163 bash[22347]: audit 2024-03-12T02:19:30.382670+0000 mon.smithi153 (mon.0) 822 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:19:31.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:30 smithi153 bash[14340]: audit 2024-03-12T02:19:30.382670+0000 mon.smithi153 (mon.0) 822 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:19:31.438 INFO:tasks.cephadm:jaeger-collector has 1/1 2024-03-12T02:19:31.438 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service... 2024-03-12T02:19:31.448 INFO:tasks.cephadm:Waiting for ceph service jaeger-query to start (timeout 300)... 2024-03-12T02:19:31.449 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:19:31.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:31 smithi163 bash[22347]: cluster 2024-03-12T02:19:30.215093+0000 mgr.smithi153.asomxr (mgr.14195) 313 : cluster 0 pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:31.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:31 smithi163 bash[22347]: audit 2024-03-12T02:19:30.628390+0000 mgr.smithi153.asomxr (mgr.14195) 314 : audit 0 from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:32.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:31 smithi153 bash[14340]: cluster 2024-03-12T02:19:30.215093+0000 mgr.smithi153.asomxr (mgr.14195) 313 : cluster 0 pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:32.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:31 smithi153 bash[14340]: audit 2024-03-12T02:19:30.628390+0000 mgr.smithi153.asomxr (mgr.14195) 314 : audit 0 from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:33.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:33 smithi163 bash[22347]: cluster 2024-03-12T02:19:32.215674+0000 mgr.smithi153.asomxr (mgr.14195) 315 : cluster 0 pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:33 smithi153 bash[14340]: cluster 2024-03-12T02:19:32.215674+0000 mgr.smithi153.asomxr (mgr.14195) 315 : cluster 0 pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:35.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:35 smithi163 bash[22347]: cluster 2024-03-12T02:19:34.216320+0000 mgr.smithi153.asomxr (mgr.14195) 316 : cluster 0 pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:36.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:35 smithi153 bash[14340]: cluster 2024-03-12T02:19:34.216320+0000 mgr.smithi153.asomxr (mgr.14195) 316 : cluster 0 pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:36.200 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:19:37.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:37 smithi153 bash[14340]: cluster 2024-03-12T02:19:36.216928+0000 mgr.smithi153.asomxr (mgr.14195) 317 : cluster 0 pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:37.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:37 smithi163 bash[22347]: cluster 2024-03-12T02:19:36.216928+0000 mgr.smithi153.asomxr (mgr.14195) 317 : cluster 0 pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:39.311 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:19:39.312 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:19:23.648341Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:19:23.647604Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:19:23.647849Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:19:23.654432Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:19:23.648584Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:19:23.649985Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:19:23.650224Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:19:23.654656Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:19:23.647352Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:19:23.647074Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:19:23.648090Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:19:23.649064Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:19:23.648824Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:19:39.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:39 smithi153 bash[14340]: cluster 2024-03-12T02:19:38.217577+0000 mgr.smithi153.asomxr (mgr.14195) 318 : cluster 0 pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:39.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:39 smithi163 bash[22347]: cluster 2024-03-12T02:19:38.217577+0000 mgr.smithi153.asomxr (mgr.14195) 318 : cluster 0 pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:40.136 INFO:tasks.cephadm:jaeger-query has 1/1 2024-03-12T02:19:40.136 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service... 2024-03-12T02:19:40.147 INFO:tasks.cephadm:Waiting for ceph service jaeger-agent to start (timeout 300)... 2024-03-12T02:19:40.147 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:19:40.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:40 smithi163 bash[22347]: audit 2024-03-12T02:19:39.296915+0000 mgr.smithi153.asomxr (mgr.14195) 319 : audit 0 from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:41.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:40 smithi153 bash[14340]: audit 2024-03-12T02:19:39.296915+0000 mgr.smithi153.asomxr (mgr.14195) 319 : audit 0 from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:41.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:41 smithi163 bash[22347]: cluster 2024-03-12T02:19:40.218190+0000 mgr.smithi153.asomxr (mgr.14195) 320 : cluster 0 pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:42.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:41 smithi153 bash[14340]: cluster 2024-03-12T02:19:40.218190+0000 mgr.smithi153.asomxr (mgr.14195) 320 : cluster 0 pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:43.978 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:43 smithi163 bash[22347]: cluster 2024-03-12T02:19:42.218752+0000 mgr.smithi153.asomxr (mgr.14195) 321 : cluster 0 pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:44.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:43 smithi153 bash[14340]: cluster 2024-03-12T02:19:42.218752+0000 mgr.smithi153.asomxr (mgr.14195) 321 : cluster 0 pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:44.901 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:19:45.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:45 smithi163 bash[22347]: cluster 2024-03-12T02:19:44.219399+0000 mgr.smithi153.asomxr (mgr.14195) 322 : cluster 0 pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:45.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:45 smithi163 bash[22347]: audit 2024-03-12T02:19:45.383388+0000 mon.smithi153 (mon.0) 823 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:19:46.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:45 smithi153 bash[14340]: cluster 2024-03-12T02:19:44.219399+0000 mgr.smithi153.asomxr (mgr.14195) 322 : cluster 0 pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:46.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:45 smithi153 bash[14340]: audit 2024-03-12T02:19:45.383388+0000 mon.smithi153 (mon.0) 823 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:19:47.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:47 smithi153 bash[14340]: cluster 2024-03-12T02:19:46.220072+0000 mgr.smithi153.asomxr (mgr.14195) 323 : cluster 0 pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:47.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:47 smithi163 bash[22347]: cluster 2024-03-12T02:19:46.220072+0000 mgr.smithi153.asomxr (mgr.14195) 323 : cluster 0 pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:48.022 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:19:48.022 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:19:23.648341Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:19:23.647604Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:19:23.647849Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:19:23.654432Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:19:23.648584Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:19:23.649985Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:19:23.650224Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:19:23.654656Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:19:23.647352Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:19:23.647074Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:19:23.648090Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:19:23.649064Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:19:23.648824Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:19:48.789 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:19:49.791 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:19:50.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:49 smithi153 bash[14340]: audit 2024-03-12T02:19:48.008612+0000 mgr.smithi153.asomxr (mgr.14195) 324 : audit 0 from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:50.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:49 smithi153 bash[14340]: cluster 2024-03-12T02:19:48.220695+0000 mgr.smithi153.asomxr (mgr.14195) 325 : cluster 0 pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:50.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:49 smithi163 bash[22347]: audit 2024-03-12T02:19:48.008612+0000 mgr.smithi153.asomxr (mgr.14195) 324 : audit 0 from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:50.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:49 smithi163 bash[22347]: cluster 2024-03-12T02:19:48.220695+0000 mgr.smithi153.asomxr (mgr.14195) 325 : cluster 0 pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:52.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:51 smithi153 bash[14340]: cluster 2024-03-12T02:19:50.221386+0000 mgr.smithi153.asomxr (mgr.14195) 326 : cluster 0 pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:52.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:51 smithi163 bash[22347]: cluster 2024-03-12T02:19:50.221386+0000 mgr.smithi153.asomxr (mgr.14195) 326 : cluster 0 pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:54.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:53 smithi153 bash[14340]: cluster 2024-03-12T02:19:52.221964+0000 mgr.smithi153.asomxr (mgr.14195) 327 : cluster 0 pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:54.199 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:53 smithi163 bash[22347]: cluster 2024-03-12T02:19:52.221964+0000 mgr.smithi153.asomxr (mgr.14195) 327 : cluster 0 pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:54.543 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:19:56.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:55 smithi153 bash[14340]: cluster 2024-03-12T02:19:54.222594+0000 mgr.smithi153.asomxr (mgr.14195) 328 : cluster 0 pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:56.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:55 smithi163 bash[22347]: cluster 2024-03-12T02:19:54.222594+0000 mgr.smithi153.asomxr (mgr.14195) 328 : cluster 0 pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:57.690 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:19:57.690 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:19:23.648341Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:19:23.647604Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:19:23.647849Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:19:23.654432Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:19:23.648584Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:19:23.649985Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:19:23.650224Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:19:23.654656Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:19:23.647352Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:19:23.647074Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:19:23.648090Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:19:23.649064Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:19:23.648824Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:19:58.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:57 smithi153 bash[14340]: cluster 2024-03-12T02:19:56.223226+0000 mgr.smithi153.asomxr (mgr.14195) 329 : cluster 0 pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:58.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:57 smithi163 bash[22347]: cluster 2024-03-12T02:19:56.223226+0000 mgr.smithi153.asomxr (mgr.14195) 329 : cluster 0 pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:19:58.493 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:19:59.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:58 smithi153 bash[14340]: audit 2024-03-12T02:19:57.676353+0000 mgr.smithi153.asomxr (mgr.14195) 330 : audit 0 from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:59.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:58 smithi163 bash[22347]: audit 2024-03-12T02:19:57.676353+0000 mgr.smithi153.asomxr (mgr.14195) 330 : audit 0 from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:19:59.494 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:20:00.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:19:59 smithi153 bash[14340]: cluster 2024-03-12T02:19:58.223857+0000 mgr.smithi153.asomxr (mgr.14195) 331 : cluster 0 pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:00.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:19:59 smithi163 bash[22347]: cluster 2024-03-12T02:19:58.223857+0000 mgr.smithi153.asomxr (mgr.14195) 331 : cluster 0 pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:01.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:00 smithi153 bash[14340]: cluster 2024-03-12T02:20:00.000123+0000 mon.smithi153 (mon.0) 824 : cluster 3 Health detail: HEALTH_WARN 2 failed cephadm daemon(s) 2024-03-12T02:20:01.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:00 smithi153 bash[14340]: cluster 2024-03-12T02:20:00.000181+0000 mon.smithi153 (mon.0) 825 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-03-12T02:20:01.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:00 smithi153 bash[14340]: cluster 2024-03-12T02:20:00.000206+0000 mon.smithi153 (mon.0) 826 : cluster 3 daemon jaeger-agent.smithi153 on smithi153 is in error state 2024-03-12T02:20:01.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:00 smithi153 bash[14340]: cluster 2024-03-12T02:20:00.000231+0000 mon.smithi153 (mon.0) 827 : cluster 3 daemon jaeger-agent.smithi163 on smithi163 is in error state 2024-03-12T02:20:01.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:00 smithi153 bash[14340]: audit 2024-03-12T02:20:00.388461+0000 mon.smithi153 (mon.0) 828 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:20:01.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:00 smithi163 bash[22347]: cluster 2024-03-12T02:20:00.000123+0000 mon.smithi153 (mon.0) 824 : cluster 3 Health detail: HEALTH_WARN 2 failed cephadm daemon(s) 2024-03-12T02:20:01.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:00 smithi163 bash[22347]: cluster 2024-03-12T02:20:00.000181+0000 mon.smithi153 (mon.0) 825 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-03-12T02:20:01.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:00 smithi163 bash[22347]: cluster 2024-03-12T02:20:00.000206+0000 mon.smithi153 (mon.0) 826 : cluster 3 daemon jaeger-agent.smithi153 on smithi153 is in error state 2024-03-12T02:20:01.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:00 smithi163 bash[22347]: cluster 2024-03-12T02:20:00.000231+0000 mon.smithi153 (mon.0) 827 : cluster 3 daemon jaeger-agent.smithi163 on smithi163 is in error state 2024-03-12T02:20:01.236 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:00 smithi163 bash[22347]: audit 2024-03-12T02:20:00.388461+0000 mon.smithi153 (mon.0) 828 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:20:02.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:01 smithi153 bash[14340]: cluster 2024-03-12T02:20:00.224673+0000 mgr.smithi153.asomxr (mgr.14195) 332 : cluster 0 pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:02.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:01 smithi163 bash[22347]: cluster 2024-03-12T02:20:00.224673+0000 mgr.smithi153.asomxr (mgr.14195) 332 : cluster 0 pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:04.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:03 smithi153 bash[14340]: cluster 2024-03-12T02:20:02.225288+0000 mgr.smithi153.asomxr (mgr.14195) 333 : cluster 0 pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:04.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:03 smithi163 bash[22347]: cluster 2024-03-12T02:20:02.225288+0000 mgr.smithi153.asomxr (mgr.14195) 333 : cluster 0 pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:04.247 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:20:06.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:05 smithi153 bash[14340]: cluster 2024-03-12T02:20:04.225844+0000 mgr.smithi153.asomxr (mgr.14195) 334 : cluster 0 pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:06.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:05 smithi163 bash[22347]: cluster 2024-03-12T02:20:04.225844+0000 mgr.smithi153.asomxr (mgr.14195) 334 : cluster 0 pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:07.559 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:20:07.559 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:19:23.648341Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:19:23.647604Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:19:23.647849Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:19:23.654432Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:19:23.648584Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:19:23.649985Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:19:23.650224Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:19:23.654656Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:19:23.647352Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:19:23.647074Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:19:23.648090Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:19:23.649064Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:19:23.648824Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:20:08.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:07 smithi153 bash[14340]: cluster 2024-03-12T02:20:06.226439+0000 mgr.smithi153.asomxr (mgr.14195) 335 : cluster 0 pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:08.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:07 smithi163 bash[22347]: cluster 2024-03-12T02:20:06.226439+0000 mgr.smithi153.asomxr (mgr.14195) 335 : cluster 0 pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:08.347 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:20:09.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:08 smithi153 bash[14340]: audit 2024-03-12T02:20:07.542426+0000 mgr.smithi153.asomxr (mgr.14195) 336 : audit 0 from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:09.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:08 smithi163 bash[22347]: audit 2024-03-12T02:20:07.542426+0000 mgr.smithi153.asomxr (mgr.14195) 336 : audit 0 from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:09.348 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:20:10.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:09 smithi153 bash[14340]: cluster 2024-03-12T02:20:08.226890+0000 mgr.smithi153.asomxr (mgr.14195) 337 : cluster 0 pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:10.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:09 smithi163 bash[22347]: cluster 2024-03-12T02:20:08.226890+0000 mgr.smithi153.asomxr (mgr.14195) 337 : cluster 0 pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:11 smithi153 bash[14340]: cluster 2024-03-12T02:20:10.227555+0000 mgr.smithi153.asomxr (mgr.14195) 338 : cluster 0 pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:12.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:11 smithi163 bash[22347]: cluster 2024-03-12T02:20:10.227555+0000 mgr.smithi153.asomxr (mgr.14195) 338 : cluster 0 pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:14.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:13 smithi153 bash[14340]: cluster 2024-03-12T02:20:12.228186+0000 mgr.smithi153.asomxr (mgr.14195) 339 : cluster 0 pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:14.104 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:20:14.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:13 smithi163 bash[22347]: cluster 2024-03-12T02:20:12.228186+0000 mgr.smithi153.asomxr (mgr.14195) 339 : cluster 0 pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:16.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:15 smithi153 bash[14340]: cluster 2024-03-12T02:20:14.228938+0000 mgr.smithi153.asomxr (mgr.14195) 340 : cluster 0 pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:16.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:15 smithi153 bash[14340]: audit 2024-03-12T02:20:15.389072+0000 mon.smithi153 (mon.0) 829 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:20:16.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:15 smithi163 bash[22347]: cluster 2024-03-12T02:20:14.228938+0000 mgr.smithi153.asomxr (mgr.14195) 340 : cluster 0 pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:16.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:15 smithi163 bash[22347]: audit 2024-03-12T02:20:15.389072+0000 mon.smithi153 (mon.0) 829 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:20:17.410 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:20:17.411 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:19:23.648341Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:19:23.647604Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:19:23.647849Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:19:23.654432Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:19:23.648584Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:19:23.649985Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:19:23.650224Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:19:23.654656Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:19:23.647352Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:19:23.647074Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:19:23.648090Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:19:23.649064Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:19:23.648824Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:20:18.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:17 smithi153 bash[14340]: cluster 2024-03-12T02:20:16.229558+0000 mgr.smithi153.asomxr (mgr.14195) 341 : cluster 0 pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:18.197 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:20:18.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:17 smithi163 bash[22347]: cluster 2024-03-12T02:20:16.229558+0000 mgr.smithi153.asomxr (mgr.14195) 341 : cluster 0 pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:19.198 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:20:19.210 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:18 smithi153 bash[14340]: audit 2024-03-12T02:20:17.395552+0000 mgr.smithi153.asomxr (mgr.14195) 342 : audit 0 from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:19.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:18 smithi163 bash[22347]: audit 2024-03-12T02:20:17.395552+0000 mgr.smithi153.asomxr (mgr.14195) 342 : audit 0 from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:20.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:19 smithi163 bash[22347]: cluster 2024-03-12T02:20:18.230188+0000 mgr.smithi153.asomxr (mgr.14195) 343 : cluster 0 pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:20.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:19 smithi153 bash[14340]: cluster 2024-03-12T02:20:18.230188+0000 mgr.smithi153.asomxr (mgr.14195) 343 : cluster 0 pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:22.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:21 smithi163 bash[22347]: cluster 2024-03-12T02:20:20.230761+0000 mgr.smithi153.asomxr (mgr.14195) 344 : cluster 0 pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:22.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:21 smithi153 bash[14340]: cluster 2024-03-12T02:20:20.230761+0000 mgr.smithi153.asomxr (mgr.14195) 344 : cluster 0 pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:23.948 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:20:24.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:23 smithi163 bash[22347]: cluster 2024-03-12T02:20:22.231354+0000 mgr.smithi153.asomxr (mgr.14195) 345 : cluster 0 pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:24.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:23 smithi153 bash[14340]: cluster 2024-03-12T02:20:22.231354+0000 mgr.smithi153.asomxr (mgr.14195) 345 : cluster 0 pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:25.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:24 smithi153 bash[14340]: audit 2024-03-12T02:20:24.313125+0000 mon.smithi153 (mon.0) 830 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:20:25.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:24 smithi153 bash[14340]: audit 2024-03-12T02:20:24.690148+0000 mon.smithi153 (mon.0) 831 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:25.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:24 smithi153 bash[14340]: audit 2024-03-12T02:20:24.697245+0000 mon.smithi153 (mon.0) 832 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:25.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:24 smithi163 bash[22347]: audit 2024-03-12T02:20:24.313125+0000 mon.smithi153 (mon.0) 830 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:20:25.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:24 smithi163 bash[22347]: audit 2024-03-12T02:20:24.690148+0000 mon.smithi153 (mon.0) 831 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:25.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:24 smithi163 bash[22347]: audit 2024-03-12T02:20:24.697245+0000 mon.smithi153 (mon.0) 832 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:26.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:25 smithi153 bash[14340]: cluster 2024-03-12T02:20:24.231810+0000 mgr.smithi153.asomxr (mgr.14195) 346 : cluster 0 pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:26.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:25 smithi163 bash[22347]: cluster 2024-03-12T02:20:24.231810+0000 mgr.smithi153.asomxr (mgr.14195) 346 : cluster 0 pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:27.069 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:20:27.069 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:19:23.648341Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:19:23.647604Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:19:23.647849Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:19:23.654432Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:19:23.648584Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:19:23.649985Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:19:23.650224Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:19:23.654656Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:19:23.647352Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:19:23.647074Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:19:23.648090Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:19:23.649064Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:19:23.648824Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:20:27.846 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:20:28.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:27 smithi153 bash[14340]: cluster 2024-03-12T02:20:26.232266+0000 mgr.smithi153.asomxr (mgr.14195) 347 : cluster 0 pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:28.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:27 smithi163 bash[22347]: cluster 2024-03-12T02:20:26.232266+0000 mgr.smithi153.asomxr (mgr.14195) 347 : cluster 0 pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:28.847 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:20:29.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:28 smithi163 bash[22347]: audit 2024-03-12T02:20:27.054752+0000 mgr.smithi153.asomxr (mgr.14195) 348 : audit 0 from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:29.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:28 smithi153 bash[14340]: audit 2024-03-12T02:20:27.054752+0000 mgr.smithi153.asomxr (mgr.14195) 348 : audit 0 from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:30.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:29 smithi163 bash[22347]: cluster 2024-03-12T02:20:28.232969+0000 mgr.smithi153.asomxr (mgr.14195) 349 : cluster 0 pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:30.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:29 smithi153 bash[14340]: cluster 2024-03-12T02:20:28.232969+0000 mgr.smithi153.asomxr (mgr.14195) 349 : cluster 0 pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:31.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:31 smithi163 bash[22347]: audit 2024-03-12T02:20:30.171960+0000 mon.smithi153 (mon.0) 833 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:31.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:31 smithi163 bash[22347]: audit 2024-03-12T02:20:30.182099+0000 mon.smithi153 (mon.0) 834 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:31.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:31 smithi163 bash[22347]: cluster 2024-03-12T02:20:30.233760+0000 mgr.smithi153.asomxr (mgr.14195) 350 : cluster 0 pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:31.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:31 smithi163 bash[22347]: audit 2024-03-12T02:20:30.389816+0000 mon.smithi153 (mon.0) 835 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:20:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:31 smithi153 bash[14340]: audit 2024-03-12T02:20:30.171960+0000 mon.smithi153 (mon.0) 833 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:31 smithi153 bash[14340]: audit 2024-03-12T02:20:30.182099+0000 mon.smithi153 (mon.0) 834 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:31 smithi153 bash[14340]: cluster 2024-03-12T02:20:30.233760+0000 mgr.smithi153.asomxr (mgr.14195) 350 : cluster 0 pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:31 smithi153 bash[14340]: audit 2024-03-12T02:20:30.389816+0000 mon.smithi153 (mon.0) 835 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:20:32.896 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:20:33.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:33 smithi163 bash[22347]: cluster 2024-03-12T02:20:32.234261+0000 mgr.smithi153.asomxr (mgr.14195) 351 : cluster 0 pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:33.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:33 smithi163 bash[22347]: audit 2024-03-12T02:20:32.395923+0000 mon.smithi153 (mon.0) 836 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:33.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:33 smithi163 bash[22347]: audit 2024-03-12T02:20:32.403793+0000 mon.smithi153 (mon.0) 837 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:33.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:33 smithi163 bash[22347]: audit 2024-03-12T02:20:32.926148+0000 mon.smithi153 (mon.0) 838 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:20:33.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:33 smithi163 bash[22347]: audit 2024-03-12T02:20:32.927115+0000 mon.smithi153 (mon.0) 839 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:20:33.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:33 smithi163 bash[22347]: cluster 2024-03-12T02:20:32.928723+0000 mgr.smithi153.asomxr (mgr.14195) 352 : cluster 0 pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:33.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:33 smithi163 bash[22347]: cluster 2024-03-12T02:20:32.929001+0000 mgr.smithi153.asomxr (mgr.14195) 353 : cluster 0 pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:33.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:33 smithi163 bash[22347]: audit 2024-03-12T02:20:32.934647+0000 mon.smithi153 (mon.0) 840 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:33.736 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:33 smithi163 bash[22347]: audit 2024-03-12T02:20:32.939529+0000 mon.smithi153 (mon.0) 841 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:20:33.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:33 smithi153 bash[14340]: cluster 2024-03-12T02:20:32.234261+0000 mgr.smithi153.asomxr (mgr.14195) 351 : cluster 0 pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:33.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:33 smithi153 bash[14340]: audit 2024-03-12T02:20:32.395923+0000 mon.smithi153 (mon.0) 836 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:33.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:33 smithi153 bash[14340]: audit 2024-03-12T02:20:32.403793+0000 mon.smithi153 (mon.0) 837 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:33.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:33 smithi153 bash[14340]: audit 2024-03-12T02:20:32.926148+0000 mon.smithi153 (mon.0) 838 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:20:33.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:33 smithi153 bash[14340]: audit 2024-03-12T02:20:32.927115+0000 mon.smithi153 (mon.0) 839 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:20:33.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:33 smithi153 bash[14340]: cluster 2024-03-12T02:20:32.928723+0000 mgr.smithi153.asomxr (mgr.14195) 352 : cluster 0 pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:33.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:33 smithi153 bash[14340]: cluster 2024-03-12T02:20:32.929001+0000 mgr.smithi153.asomxr (mgr.14195) 353 : cluster 0 pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:33.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:33 smithi153 bash[14340]: audit 2024-03-12T02:20:32.934647+0000 mon.smithi153 (mon.0) 840 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:20:33.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:33 smithi153 bash[14340]: audit 2024-03-12T02:20:32.939529+0000 mon.smithi153 (mon.0) 841 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:20:35.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:35 smithi153 bash[14340]: cluster 2024-03-12T02:20:34.929597+0000 mgr.smithi153.asomxr (mgr.14195) 354 : cluster 0 pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:35.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:35 smithi163 bash[22347]: cluster 2024-03-12T02:20:34.929597+0000 mgr.smithi153.asomxr (mgr.14195) 354 : cluster 0 pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:36.300 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:20:36.300 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:30.156799Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:30.157062Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:20:30.159043Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:30.158817Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:20:30.159269Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:30.157558Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:30.157784Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:30.157326Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:30.158008Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:20:37.059 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:20:37.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:37 smithi153 bash[14340]: audit 2024-03-12T02:20:36.286448+0000 mgr.smithi153.asomxr (mgr.14195) 355 : audit 0 from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:37.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:37 smithi153 bash[14340]: cluster 2024-03-12T02:20:36.930206+0000 mgr.smithi153.asomxr (mgr.14195) 356 : cluster 0 pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:37.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:37 smithi163 bash[22347]: audit 2024-03-12T02:20:36.286448+0000 mgr.smithi153.asomxr (mgr.14195) 355 : audit 0 from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:37.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:37 smithi163 bash[22347]: cluster 2024-03-12T02:20:36.930206+0000 mgr.smithi153.asomxr (mgr.14195) 356 : cluster 0 pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:38.060 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:20:39.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:39 smithi153 bash[14340]: cluster 2024-03-12T02:20:38.930795+0000 mgr.smithi153.asomxr (mgr.14195) 357 : cluster 0 pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:39.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:39 smithi163 bash[22347]: cluster 2024-03-12T02:20:38.930795+0000 mgr.smithi153.asomxr (mgr.14195) 357 : cluster 0 pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:41.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:41 smithi153 bash[14340]: cluster 2024-03-12T02:20:40.931475+0000 mgr.smithi153.asomxr (mgr.14195) 358 : cluster 0 pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:41.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:41 smithi163 bash[22347]: cluster 2024-03-12T02:20:40.931475+0000 mgr.smithi153.asomxr (mgr.14195) 358 : cluster 0 pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:42.816 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:20:43.953 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:43 smithi153 bash[14340]: cluster 2024-03-12T02:20:42.932138+0000 mgr.smithi153.asomxr (mgr.14195) 359 : cluster 0 pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:43.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:43 smithi163 bash[22347]: cluster 2024-03-12T02:20:42.932138+0000 mgr.smithi153.asomxr (mgr.14195) 359 : cluster 0 pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:45.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:45 smithi153 bash[14340]: cluster 2024-03-12T02:20:44.932762+0000 mgr.smithi153.asomxr (mgr.14195) 360 : cluster 0 pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:45.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:45 smithi163 bash[22347]: cluster 2024-03-12T02:20:44.932762+0000 mgr.smithi153.asomxr (mgr.14195) 360 : cluster 0 pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:46.192 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:20:46.192 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:30.156799Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:30.157062Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:20:30.159043Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:30.158817Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:20:30.159269Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:30.157558Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:30.157784Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:30.157326Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:30.158008Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:20:46.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:46 smithi153 bash[14340]: audit 2024-03-12T02:20:45.390296+0000 mon.smithi153 (mon.0) 842 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:20:46.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:46 smithi163 bash[22347]: audit 2024-03-12T02:20:45.390296+0000 mon.smithi153 (mon.0) 842 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:20:46.918 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:20:47.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:47 smithi153 bash[14340]: audit 2024-03-12T02:20:46.176854+0000 mgr.smithi153.asomxr (mgr.14195) 361 : audit 0 from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:47.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:47 smithi153 bash[14340]: cluster 2024-03-12T02:20:46.933410+0000 mgr.smithi153.asomxr (mgr.14195) 362 : cluster 0 pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:47.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:47 smithi163 bash[22347]: audit 2024-03-12T02:20:46.176854+0000 mgr.smithi153.asomxr (mgr.14195) 361 : audit 0 from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:47.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:47 smithi163 bash[22347]: cluster 2024-03-12T02:20:46.933410+0000 mgr.smithi153.asomxr (mgr.14195) 362 : cluster 0 pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:47.919 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:20:49.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:49 smithi153 bash[14340]: cluster 2024-03-12T02:20:48.934020+0000 mgr.smithi153.asomxr (mgr.14195) 363 : cluster 0 pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:49.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:49 smithi163 bash[22347]: cluster 2024-03-12T02:20:48.934020+0000 mgr.smithi153.asomxr (mgr.14195) 363 : cluster 0 pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:51.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:51 smithi153 bash[14340]: cluster 2024-03-12T02:20:50.934625+0000 mgr.smithi153.asomxr (mgr.14195) 364 : cluster 0 pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:51.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:51 smithi163 bash[22347]: cluster 2024-03-12T02:20:50.934625+0000 mgr.smithi153.asomxr (mgr.14195) 364 : cluster 0 pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:52.675 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:20:53.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:53 smithi163 bash[22347]: cluster 2024-03-12T02:20:52.935225+0000 mgr.smithi153.asomxr (mgr.14195) 365 : cluster 0 pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:54.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:53 smithi153 bash[14340]: cluster 2024-03-12T02:20:52.935225+0000 mgr.smithi153.asomxr (mgr.14195) 365 : cluster 0 pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:55.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:55 smithi153 bash[14340]: cluster 2024-03-12T02:20:54.935846+0000 mgr.smithi153.asomxr (mgr.14195) 366 : cluster 0 pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:55.450 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:55 smithi163 bash[22347]: cluster 2024-03-12T02:20:54.935846+0000 mgr.smithi153.asomxr (mgr.14195) 366 : cluster 0 pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:56.007 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:20:56.008 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:30.156799Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:30.157062Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:20:30.159043Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:30.158817Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:20:30.159269Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:30.157558Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:30.157784Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:30.157326Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:30.158008Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:20:56.789 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:20:57.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:57 smithi153 bash[14340]: audit 2024-03-12T02:20:55.992470+0000 mgr.smithi153.asomxr (mgr.14195) 367 : audit 0 from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:57.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:57 smithi153 bash[14340]: cluster 2024-03-12T02:20:56.936469+0000 mgr.smithi153.asomxr (mgr.14195) 368 : cluster 0 pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:57.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:57 smithi163 bash[22347]: audit 2024-03-12T02:20:55.992470+0000 mgr.smithi153.asomxr (mgr.14195) 367 : audit 0 from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:20:57.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:57 smithi163 bash[22347]: cluster 2024-03-12T02:20:56.936469+0000 mgr.smithi153.asomxr (mgr.14195) 368 : cluster 0 pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:57.791 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:20:59.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:20:59 smithi153 bash[14340]: cluster 2024-03-12T02:20:58.937098+0000 mgr.smithi153.asomxr (mgr.14195) 369 : cluster 0 pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:20:59.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:20:59 smithi163 bash[22347]: cluster 2024-03-12T02:20:58.937098+0000 mgr.smithi153.asomxr (mgr.14195) 369 : cluster 0 pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:00.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:00 smithi163 bash[22347]: audit 2024-03-12T02:21:00.390515+0000 mon.smithi153 (mon.0) 843 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:21:00.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:00 smithi153 bash[14340]: audit 2024-03-12T02:21:00.390515+0000 mon.smithi153 (mon.0) 843 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:21:01.735 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:01 smithi163 bash[22347]: cluster 2024-03-12T02:21:00.937698+0000 mgr.smithi153.asomxr (mgr.14195) 370 : cluster 0 pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:01.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:01 smithi153 bash[14340]: cluster 2024-03-12T02:21:00.937698+0000 mgr.smithi153.asomxr (mgr.14195) 370 : cluster 0 pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:02.550 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:21:03.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:03 smithi163 bash[22347]: cluster 2024-03-12T02:21:02.938333+0000 mgr.smithi153.asomxr (mgr.14195) 371 : cluster 0 pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:04.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:03 smithi153 bash[14340]: cluster 2024-03-12T02:21:02.938333+0000 mgr.smithi153.asomxr (mgr.14195) 371 : cluster 0 pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:05.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:05 smithi153 bash[14340]: cluster 2024-03-12T02:21:04.938995+0000 mgr.smithi153.asomxr (mgr.14195) 372 : cluster 0 pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:05.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:05 smithi163 bash[22347]: cluster 2024-03-12T02:21:04.938995+0000 mgr.smithi153.asomxr (mgr.14195) 372 : cluster 0 pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:05.628 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:21:05.628 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:30.156799Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:30.157062Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:20:30.159043Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:30.158817Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:20:30.159269Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:30.157558Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:30.157784Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:30.157326Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:30.158008Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:21:06.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:06 smithi153 bash[14340]: audit 2024-03-12T02:21:05.613794+0000 mgr.smithi153.asomxr (mgr.14195) 373 : audit 0 from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:06.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:06 smithi163 bash[22347]: audit 2024-03-12T02:21:05.613794+0000 mgr.smithi153.asomxr (mgr.14195) 373 : audit 0 from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:06.494 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:21:07.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:07 smithi153 bash[14340]: cluster 2024-03-12T02:21:06.939632+0000 mgr.smithi153.asomxr (mgr.14195) 374 : cluster 0 pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:07.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:07 smithi163 bash[22347]: cluster 2024-03-12T02:21:06.939632+0000 mgr.smithi153.asomxr (mgr.14195) 374 : cluster 0 pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:07.495 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:21:09.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:09 smithi153 bash[14340]: cluster 2024-03-12T02:21:08.940192+0000 mgr.smithi153.asomxr (mgr.14195) 375 : cluster 0 pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:09.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:09 smithi163 bash[22347]: cluster 2024-03-12T02:21:08.940192+0000 mgr.smithi153.asomxr (mgr.14195) 375 : cluster 0 pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:11.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:11 smithi153 bash[14340]: cluster 2024-03-12T02:21:10.940884+0000 mgr.smithi153.asomxr (mgr.14195) 376 : cluster 0 pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:11.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:11 smithi163 bash[22347]: cluster 2024-03-12T02:21:10.940884+0000 mgr.smithi153.asomxr (mgr.14195) 376 : cluster 0 pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:12.250 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:21:13.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:13 smithi153 bash[14340]: cluster 2024-03-12T02:21:12.941636+0000 mgr.smithi153.asomxr (mgr.14195) 377 : cluster 0 pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:13.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:13 smithi163 bash[22347]: cluster 2024-03-12T02:21:12.941636+0000 mgr.smithi153.asomxr (mgr.14195) 377 : cluster 0 pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:15.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:15 smithi153 bash[14340]: cluster 2024-03-12T02:21:14.942400+0000 mgr.smithi153.asomxr (mgr.14195) 378 : cluster 0 pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:15.469 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:21:15.469 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:30.156799Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:30.157062Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:20:30.159043Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:30.158817Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:20:30.159269Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:30.157558Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:30.157784Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:30.157326Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:30.158008Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:21:15.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:15 smithi163 bash[22347]: cluster 2024-03-12T02:21:14.942400+0000 mgr.smithi153.asomxr (mgr.14195) 378 : cluster 0 pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:16.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:16 smithi163 bash[22347]: audit 2024-03-12T02:21:15.390187+0000 mon.smithi153 (mon.0) 844 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:21:16.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:16 smithi163 bash[22347]: audit 2024-03-12T02:21:15.454802+0000 mgr.smithi153.asomxr (mgr.14195) 379 : audit 0 from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:16.249 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:21:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:16 smithi153 bash[14340]: audit 2024-03-12T02:21:15.390187+0000 mon.smithi153 (mon.0) 844 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:21:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:16 smithi153 bash[14340]: audit 2024-03-12T02:21:15.454802+0000 mgr.smithi153.asomxr (mgr.14195) 379 : audit 0 from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:17.249 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:21:17.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:17 smithi153 bash[14340]: cluster 2024-03-12T02:21:16.943047+0000 mgr.smithi153.asomxr (mgr.14195) 380 : cluster 0 pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:17.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:17 smithi163 bash[22347]: cluster 2024-03-12T02:21:16.943047+0000 mgr.smithi153.asomxr (mgr.14195) 380 : cluster 0 pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:19.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:19 smithi153 bash[14340]: cluster 2024-03-12T02:21:18.943674+0000 mgr.smithi153.asomxr (mgr.14195) 381 : cluster 0 pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:19.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:19 smithi163 bash[22347]: cluster 2024-03-12T02:21:18.943674+0000 mgr.smithi153.asomxr (mgr.14195) 381 : cluster 0 pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:21.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:21 smithi163 bash[22347]: cluster 2024-03-12T02:21:20.944245+0000 mgr.smithi153.asomxr (mgr.14195) 382 : cluster 0 pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:21.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:21 smithi153 bash[14340]: cluster 2024-03-12T02:21:20.944245+0000 mgr.smithi153.asomxr (mgr.14195) 382 : cluster 0 pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:21.997 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:21:23.985 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:23 smithi163 bash[22347]: cluster 2024-03-12T02:21:22.944842+0000 mgr.smithi153.asomxr (mgr.14195) 383 : cluster 0 pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:24.004 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:23 smithi153 bash[14340]: cluster 2024-03-12T02:21:22.944842+0000 mgr.smithi153.asomxr (mgr.14195) 383 : cluster 0 pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:25.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:25 smithi153 bash[14340]: cluster 2024-03-12T02:21:24.945460+0000 mgr.smithi153.asomxr (mgr.14195) 384 : cluster 0 pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:25.456 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:21:25.456 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:30.156799Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:30.157062Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:20:30.159043Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:30.158817Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:20:30.159269Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:30.157558Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:30.157784Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:30.157326Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:30.158008Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:21:25.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:25 smithi163 bash[22347]: cluster 2024-03-12T02:21:24.945460+0000 mgr.smithi153.asomxr (mgr.14195) 384 : cluster 0 pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:26.269 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:21:26.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:26 smithi153 bash[14340]: audit 2024-03-12T02:21:25.443261+0000 mgr.smithi153.asomxr (mgr.14195) 385 : audit 0 from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:26.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:26 smithi163 bash[22347]: audit 2024-03-12T02:21:25.443261+0000 mgr.smithi153.asomxr (mgr.14195) 385 : audit 0 from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:27.269 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:21:27.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:27 smithi153 bash[14340]: cluster 2024-03-12T02:21:26.946032+0000 mgr.smithi153.asomxr (mgr.14195) 386 : cluster 0 pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:27.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:27 smithi163 bash[22347]: cluster 2024-03-12T02:21:26.946032+0000 mgr.smithi153.asomxr (mgr.14195) 386 : cluster 0 pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:29.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:29 smithi153 bash[14340]: cluster 2024-03-12T02:21:28.946603+0000 mgr.smithi153.asomxr (mgr.14195) 387 : cluster 0 pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:29.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:29 smithi163 bash[22347]: cluster 2024-03-12T02:21:28.946603+0000 mgr.smithi153.asomxr (mgr.14195) 387 : cluster 0 pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:30.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:30 smithi163 bash[22347]: audit 2024-03-12T02:21:30.391184+0000 mon.smithi153 (mon.0) 845 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:21:30.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:30 smithi153 bash[14340]: audit 2024-03-12T02:21:30.391184+0000 mon.smithi153 (mon.0) 845 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:21:31.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:31 smithi163 bash[22347]: cluster 2024-03-12T02:21:30.947224+0000 mgr.smithi153.asomxr (mgr.14195) 388 : cluster 0 pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:31.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:31 smithi153 bash[14340]: cluster 2024-03-12T02:21:30.947224+0000 mgr.smithi153.asomxr (mgr.14195) 388 : cluster 0 pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:32.022 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:21:33.983 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:33 smithi153 bash[14340]: cluster 2024-03-12T02:21:32.947869+0000 mgr.smithi153.asomxr (mgr.14195) 389 : cluster 0 pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:33.983 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:33 smithi153 bash[14340]: audit 2024-03-12T02:21:33.025094+0000 mon.smithi153 (mon.0) 846 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:21:33.984 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:33 smithi163 bash[22347]: cluster 2024-03-12T02:21:32.947869+0000 mgr.smithi153.asomxr (mgr.14195) 389 : cluster 0 pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:33.984 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:33 smithi163 bash[22347]: audit 2024-03-12T02:21:33.025094+0000 mon.smithi153 (mon.0) 846 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:21:35.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:35 smithi153 bash[14340]: cluster 2024-03-12T02:21:34.948508+0000 mgr.smithi153.asomxr (mgr.14195) 390 : cluster 0 pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:35.384 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:21:35.384 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:30.156799Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:30.157062Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:20:30.159043Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:30.158817Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:20:30.159269Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:30.157558Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:30.157784Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:30.157326Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:30.158008Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:21:35.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:35 smithi163 bash[22347]: cluster 2024-03-12T02:21:34.948508+0000 mgr.smithi153.asomxr (mgr.14195) 390 : cluster 0 pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:36.170 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:21:36.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:36 smithi153 bash[14340]: audit 2024-03-12T02:21:35.367800+0000 mgr.smithi153.asomxr (mgr.14195) 391 : audit 0 from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:36.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:36 smithi163 bash[22347]: audit 2024-03-12T02:21:35.367800+0000 mgr.smithi153.asomxr (mgr.14195) 391 : audit 0 from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:37.171 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:21:37.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:37 smithi153 bash[14340]: cluster 2024-03-12T02:21:36.949060+0000 mgr.smithi153.asomxr (mgr.14195) 392 : cluster 0 pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:37.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:37 smithi163 bash[22347]: cluster 2024-03-12T02:21:36.949060+0000 mgr.smithi153.asomxr (mgr.14195) 392 : cluster 0 pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:40.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:39 smithi163 bash[22347]: audit 2024-03-12T02:21:38.876523+0000 mon.smithi153 (mon.0) 847 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:21:40.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:39 smithi163 bash[22347]: audit 2024-03-12T02:21:38.885710+0000 mon.smithi153 (mon.0) 848 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:21:40.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:39 smithi163 bash[22347]: cluster 2024-03-12T02:21:38.949709+0000 mgr.smithi153.asomxr (mgr.14195) 393 : cluster 0 pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:40.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:39 smithi163 bash[22347]: audit 2024-03-12T02:21:39.359009+0000 mon.smithi153 (mon.0) 849 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:21:40.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:39 smithi163 bash[22347]: audit 2024-03-12T02:21:39.360629+0000 mon.smithi153 (mon.0) 850 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:21:40.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:39 smithi163 bash[22347]: audit 2024-03-12T02:21:39.376353+0000 mon.smithi153 (mon.0) 851 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:21:40.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:39 smithi163 bash[22347]: audit 2024-03-12T02:21:39.381090+0000 mon.smithi153 (mon.0) 852 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:21:40.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:39 smithi153 bash[14340]: audit 2024-03-12T02:21:38.876523+0000 mon.smithi153 (mon.0) 847 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:21:40.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:39 smithi153 bash[14340]: audit 2024-03-12T02:21:38.885710+0000 mon.smithi153 (mon.0) 848 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:21:40.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:39 smithi153 bash[14340]: cluster 2024-03-12T02:21:38.949709+0000 mgr.smithi153.asomxr (mgr.14195) 393 : cluster 0 pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:40.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:39 smithi153 bash[14340]: audit 2024-03-12T02:21:39.359009+0000 mon.smithi153 (mon.0) 849 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:21:40.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:39 smithi153 bash[14340]: audit 2024-03-12T02:21:39.360629+0000 mon.smithi153 (mon.0) 850 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:21:40.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:39 smithi153 bash[14340]: audit 2024-03-12T02:21:39.376353+0000 mon.smithi153 (mon.0) 851 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:21:40.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:39 smithi153 bash[14340]: audit 2024-03-12T02:21:39.381090+0000 mon.smithi153 (mon.0) 852 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:21:41.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:40 smithi163 bash[22347]: cluster 2024-03-12T02:21:39.363611+0000 mgr.smithi153.asomxr (mgr.14195) 394 : cluster 0 pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:41.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:40 smithi153 bash[14340]: cluster 2024-03-12T02:21:39.363611+0000 mgr.smithi153.asomxr (mgr.14195) 394 : cluster 0 pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:41.930 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:21:43.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:42 smithi163 bash[22347]: cluster 2024-03-12T02:21:41.365161+0000 mgr.smithi153.asomxr (mgr.14195) 395 : cluster 0 pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:43.277 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:42 smithi153 bash[14340]: cluster 2024-03-12T02:21:41.365161+0000 mgr.smithi153.asomxr (mgr.14195) 395 : cluster 0 pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:45.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:44 smithi153 bash[14340]: cluster 2024-03-12T02:21:43.365614+0000 mgr.smithi153.asomxr (mgr.14195) 396 : cluster 0 pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:45.220 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:21:45.220 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:21:38.865003Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:21:38.865255Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:21:45.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:44 smithi163 bash[22347]: cluster 2024-03-12T02:21:43.365614+0000 mgr.smithi153.asomxr (mgr.14195) 396 : cluster 0 pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:46.045 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:21:46.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:45 smithi163 bash[22347]: audit 2024-03-12T02:21:45.391454+0000 mon.smithi153 (mon.0) 853 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:21:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:45 smithi153 bash[14340]: audit 2024-03-12T02:21:45.391454+0000 mon.smithi153 (mon.0) 853 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:21:47.046 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:21:47.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:46 smithi163 bash[22347]: audit 2024-03-12T02:21:45.204058+0000 mgr.smithi153.asomxr (mgr.14195) 397 : audit 0 from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:47.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:46 smithi163 bash[22347]: cluster 2024-03-12T02:21:45.366299+0000 mgr.smithi153.asomxr (mgr.14195) 398 : cluster 0 pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:47.326 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:46 smithi153 bash[14340]: audit 2024-03-12T02:21:45.204058+0000 mgr.smithi153.asomxr (mgr.14195) 397 : audit 0 from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:47.326 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:46 smithi153 bash[14340]: cluster 2024-03-12T02:21:45.366299+0000 mgr.smithi153.asomxr (mgr.14195) 398 : cluster 0 pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:48 smithi163 bash[22347]: cluster 2024-03-12T02:21:47.366941+0000 mgr.smithi153.asomxr (mgr.14195) 399 : cluster 0 pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:49.270 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:48 smithi153 bash[14340]: cluster 2024-03-12T02:21:47.366941+0000 mgr.smithi153.asomxr (mgr.14195) 399 : cluster 0 pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:51.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:50 smithi163 bash[22347]: cluster 2024-03-12T02:21:49.367570+0000 mgr.smithi153.asomxr (mgr.14195) 400 : cluster 0 pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:51.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:50 smithi153 bash[14340]: cluster 2024-03-12T02:21:49.367570+0000 mgr.smithi153.asomxr (mgr.14195) 400 : cluster 0 pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:51.801 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:21:53.191 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:52 smithi153 bash[14340]: cluster 2024-03-12T02:21:51.369847+0000 mgr.smithi153.asomxr (mgr.14195) 401 : cluster 0 pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:53.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:52 smithi163 bash[22347]: cluster 2024-03-12T02:21:51.369847+0000 mgr.smithi153.asomxr (mgr.14195) 401 : cluster 0 pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:55.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:54 smithi153 bash[14340]: cluster 2024-03-12T02:21:53.370265+0000 mgr.smithi153.asomxr (mgr.14195) 402 : cluster 0 pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:55.152 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:21:55.152 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:21:38.865003Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:21:38.865255Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:21:55.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:54 smithi163 bash[22347]: cluster 2024-03-12T02:21:53.370265+0000 mgr.smithi153.asomxr (mgr.14195) 402 : cluster 0 pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:55.984 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:21:56.985 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:21:57.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:56 smithi163 bash[22347]: audit 2024-03-12T02:21:55.135417+0000 mgr.smithi153.asomxr (mgr.14195) 403 : audit 0 from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:57.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:56 smithi163 bash[22347]: cluster 2024-03-12T02:21:55.370935+0000 mgr.smithi153.asomxr (mgr.14195) 404 : cluster 0 pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:57.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:56 smithi153 bash[14340]: audit 2024-03-12T02:21:55.135417+0000 mgr.smithi153.asomxr (mgr.14195) 403 : audit 0 from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:21:57.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:56 smithi153 bash[14340]: cluster 2024-03-12T02:21:55.370935+0000 mgr.smithi153.asomxr (mgr.14195) 404 : cluster 0 pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:59.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:21:58 smithi163 bash[22347]: cluster 2024-03-12T02:21:57.371537+0000 mgr.smithi153.asomxr (mgr.14195) 405 : cluster 0 pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:21:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:21:58 smithi153 bash[14340]: cluster 2024-03-12T02:21:57.371537+0000 mgr.smithi153.asomxr (mgr.14195) 405 : cluster 0 pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:01.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:00 smithi163 bash[22347]: cluster 2024-03-12T02:21:59.372158+0000 mgr.smithi153.asomxr (mgr.14195) 406 : cluster 0 pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:01.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:00 smithi163 bash[22347]: audit 2024-03-12T02:22:00.391178+0000 mon.smithi153 (mon.0) 854 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:22:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:00 smithi153 bash[14340]: cluster 2024-03-12T02:21:59.372158+0000 mgr.smithi153.asomxr (mgr.14195) 406 : cluster 0 pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:00 smithi153 bash[14340]: audit 2024-03-12T02:22:00.391178+0000 mon.smithi153 (mon.0) 854 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:22:01.734 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:22:03.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:02 smithi153 bash[14340]: cluster 2024-03-12T02:22:01.372773+0000 mgr.smithi153.asomxr (mgr.14195) 407 : cluster 0 pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:03.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:02 smithi163 bash[22347]: cluster 2024-03-12T02:22:01.372773+0000 mgr.smithi153.asomxr (mgr.14195) 407 : cluster 0 pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:04.984 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:22:04.984 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:21:38.865003Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:21:38.865255Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:22:05.206 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:04 smithi153 bash[14340]: cluster 2024-03-12T02:22:03.373370+0000 mgr.smithi153.asomxr (mgr.14195) 408 : cluster 0 pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:05.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:04 smithi163 bash[22347]: cluster 2024-03-12T02:22:03.373370+0000 mgr.smithi153.asomxr (mgr.14195) 408 : cluster 0 pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:05.771 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:22:06.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:05 smithi153 bash[14340]: audit 2024-03-12T02:22:04.973740+0000 mgr.smithi153.asomxr (mgr.14195) 409 : audit 0 from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:06.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:05 smithi163 bash[22347]: audit 2024-03-12T02:22:04.973740+0000 mgr.smithi153.asomxr (mgr.14195) 409 : audit 0 from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:06.772 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:22:07.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:06 smithi153 bash[14340]: cluster 2024-03-12T02:22:05.374004+0000 mgr.smithi153.asomxr (mgr.14195) 410 : cluster 0 pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:07.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:06 smithi163 bash[22347]: cluster 2024-03-12T02:22:05.374004+0000 mgr.smithi153.asomxr (mgr.14195) 410 : cluster 0 pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:09.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:08 smithi163 bash[22347]: cluster 2024-03-12T02:22:07.374603+0000 mgr.smithi153.asomxr (mgr.14195) 411 : cluster 0 pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:09.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:08 smithi153 bash[14340]: cluster 2024-03-12T02:22:07.374603+0000 mgr.smithi153.asomxr (mgr.14195) 411 : cluster 0 pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:11.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:10 smithi163 bash[22347]: cluster 2024-03-12T02:22:09.375075+0000 mgr.smithi153.asomxr (mgr.14195) 412 : cluster 0 pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:10 smithi153 bash[14340]: cluster 2024-03-12T02:22:09.375075+0000 mgr.smithi153.asomxr (mgr.14195) 412 : cluster 0 pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:11.529 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:22:13.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:12 smithi163 bash[22347]: cluster 2024-03-12T02:22:11.375646+0000 mgr.smithi153.asomxr (mgr.14195) 413 : cluster 0 pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:13.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:12 smithi153 bash[14340]: cluster 2024-03-12T02:22:11.375646+0000 mgr.smithi153.asomxr (mgr.14195) 413 : cluster 0 pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:14.838 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:22:14.839 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:21:38.865003Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:21:38.865255Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:22:15.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:14 smithi153 bash[14340]: cluster 2024-03-12T02:22:13.376333+0000 mgr.smithi153.asomxr (mgr.14195) 414 : cluster 0 pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:15.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:14 smithi163 bash[22347]: cluster 2024-03-12T02:22:13.376333+0000 mgr.smithi153.asomxr (mgr.14195) 414 : cluster 0 pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:15.626 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:22:16.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:15 smithi163 bash[22347]: audit 2024-03-12T02:22:14.823537+0000 mgr.smithi153.asomxr (mgr.14195) 415 : audit 0 from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:16.238 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:15 smithi163 bash[22347]: audit 2024-03-12T02:22:15.392273+0000 mon.smithi153 (mon.0) 855 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:22:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:15 smithi153 bash[14340]: audit 2024-03-12T02:22:14.823537+0000 mgr.smithi153.asomxr (mgr.14195) 415 : audit 0 from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:15 smithi153 bash[14340]: audit 2024-03-12T02:22:15.392273+0000 mon.smithi153 (mon.0) 855 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:22:16.627 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:22:17.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:16 smithi163 bash[22347]: cluster 2024-03-12T02:22:15.377046+0000 mgr.smithi153.asomxr (mgr.14195) 416 : cluster 0 pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:18.984 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:18 smithi163 bash[22347]: cluster 2024-03-12T02:22:17.377676+0000 mgr.smithi153.asomxr (mgr.14195) 417 : cluster 0 pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:19.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:16 smithi153 bash[14340]: cluster 2024-03-12T02:22:15.377046+0000 mgr.smithi153.asomxr (mgr.14195) 416 : cluster 0 pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:19.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:18 smithi153 bash[14340]: cluster 2024-03-12T02:22:17.377676+0000 mgr.smithi153.asomxr (mgr.14195) 417 : cluster 0 pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:20.984 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:20 smithi163 bash[22347]: cluster 2024-03-12T02:22:19.379060+0000 mgr.smithi153.asomxr (mgr.14195) 418 : cluster 0 pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:21.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:20 smithi153 bash[14340]: cluster 2024-03-12T02:22:19.379060+0000 mgr.smithi153.asomxr (mgr.14195) 418 : cluster 0 pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:21.386 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:22:23.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:22 smithi153 bash[14340]: cluster 2024-03-12T02:22:21.379762+0000 mgr.smithi153.asomxr (mgr.14195) 419 : cluster 0 pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:23.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:22 smithi163 bash[22347]: cluster 2024-03-12T02:22:21.379762+0000 mgr.smithi153.asomxr (mgr.14195) 419 : cluster 0 pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:24.734 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:22:24.734 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:21:38.865003Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:21:38.865255Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:22:25.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:24 smithi153 bash[14340]: cluster 2024-03-12T02:22:23.380440+0000 mgr.smithi153.asomxr (mgr.14195) 420 : cluster 0 pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:25.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:24 smithi163 bash[22347]: cluster 2024-03-12T02:22:23.380440+0000 mgr.smithi153.asomxr (mgr.14195) 420 : cluster 0 pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:25.469 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:22:26.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:25 smithi153 bash[14340]: audit 2024-03-12T02:22:24.718465+0000 mgr.smithi153.asomxr (mgr.14195) 421 : audit 0 from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:26.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:25 smithi163 bash[22347]: audit 2024-03-12T02:22:24.718465+0000 mgr.smithi153.asomxr (mgr.14195) 421 : audit 0 from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:26.471 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:22:27.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:26 smithi153 bash[14340]: cluster 2024-03-12T02:22:25.380995+0000 mgr.smithi153.asomxr (mgr.14195) 422 : cluster 0 pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:27.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:26 smithi163 bash[22347]: cluster 2024-03-12T02:22:25.380995+0000 mgr.smithi153.asomxr (mgr.14195) 422 : cluster 0 pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:29.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:28 smithi153 bash[14340]: cluster 2024-03-12T02:22:27.381614+0000 mgr.smithi153.asomxr (mgr.14195) 423 : cluster 0 pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:29.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:28 smithi163 bash[22347]: cluster 2024-03-12T02:22:27.381614+0000 mgr.smithi153.asomxr (mgr.14195) 423 : cluster 0 pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:31.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:30 smithi153 bash[14340]: cluster 2024-03-12T02:22:29.382268+0000 mgr.smithi153.asomxr (mgr.14195) 424 : cluster 0 pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:31.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:30 smithi153 bash[14340]: audit 2024-03-12T02:22:30.392992+0000 mon.smithi153 (mon.0) 856 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:22:31.229 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:22:31.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:30 smithi163 bash[22347]: cluster 2024-03-12T02:22:29.382268+0000 mgr.smithi153.asomxr (mgr.14195) 424 : cluster 0 pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:31.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:30 smithi163 bash[22347]: audit 2024-03-12T02:22:30.392992+0000 mon.smithi153 (mon.0) 856 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:22:32.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:32 smithi153 bash[14340]: cluster 2024-03-12T02:22:31.382848+0000 mgr.smithi153.asomxr (mgr.14195) 425 : cluster 0 pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:33.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:32 smithi163 bash[22347]: cluster 2024-03-12T02:22:31.382848+0000 mgr.smithi153.asomxr (mgr.14195) 425 : cluster 0 pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:34.582 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:22:34.582 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:21:38.865003Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:21:38.865255Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:22:35.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:34 smithi153 bash[14340]: cluster 2024-03-12T02:22:33.383445+0000 mgr.smithi153.asomxr (mgr.14195) 426 : cluster 0 pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:35.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:34 smithi163 bash[22347]: cluster 2024-03-12T02:22:33.383445+0000 mgr.smithi153.asomxr (mgr.14195) 426 : cluster 0 pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:35.403 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:22:36.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:35 smithi153 bash[14340]: audit 2024-03-12T02:22:34.566502+0000 mgr.smithi153.asomxr (mgr.14195) 427 : audit 0 from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:36.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:35 smithi163 bash[22347]: audit 2024-03-12T02:22:34.566502+0000 mgr.smithi153.asomxr (mgr.14195) 427 : audit 0 from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:36.405 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:22:37.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:36 smithi153 bash[14340]: cluster 2024-03-12T02:22:35.384097+0000 mgr.smithi153.asomxr (mgr.14195) 428 : cluster 0 pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:37.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:36 smithi163 bash[22347]: cluster 2024-03-12T02:22:35.384097+0000 mgr.smithi153.asomxr (mgr.14195) 428 : cluster 0 pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:39.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:38 smithi153 bash[14340]: cluster 2024-03-12T02:22:37.384758+0000 mgr.smithi153.asomxr (mgr.14195) 429 : cluster 0 pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:39.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:38 smithi163 bash[22347]: cluster 2024-03-12T02:22:37.384758+0000 mgr.smithi153.asomxr (mgr.14195) 429 : cluster 0 pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:40.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:39 smithi153 bash[14340]: audit 2024-03-12T02:22:39.469922+0000 mon.smithi153 (mon.0) 857 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:22:40.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:39 smithi163 bash[22347]: audit 2024-03-12T02:22:39.469922+0000 mon.smithi153 (mon.0) 857 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:22:41.159 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:22:41.191 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:40 smithi153 bash[14340]: cluster 2024-03-12T02:22:39.385453+0000 mgr.smithi153.asomxr (mgr.14195) 430 : cluster 0 pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:41.191 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:40 smithi153 bash[14340]: audit 2024-03-12T02:22:39.850170+0000 mon.smithi153 (mon.0) 858 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:22:41.191 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:40 smithi153 bash[14340]: audit 2024-03-12T02:22:39.860853+0000 mon.smithi153 (mon.0) 859 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:22:41.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:40 smithi163 bash[22347]: cluster 2024-03-12T02:22:39.385453+0000 mgr.smithi153.asomxr (mgr.14195) 430 : cluster 0 pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:41.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:40 smithi163 bash[22347]: audit 2024-03-12T02:22:39.850170+0000 mon.smithi153 (mon.0) 858 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:22:41.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:40 smithi163 bash[22347]: audit 2024-03-12T02:22:39.860853+0000 mon.smithi153 (mon.0) 859 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:22:43.228 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:42 smithi153 bash[14340]: cluster 2024-03-12T02:22:41.385894+0000 mgr.smithi153.asomxr (mgr.14195) 431 : cluster 0 pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:43.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:42 smithi163 bash[22347]: cluster 2024-03-12T02:22:41.385894+0000 mgr.smithi153.asomxr (mgr.14195) 431 : cluster 0 pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:44.752 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:22:44.752 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:21:38.865003Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:21:38.865255Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:22:45.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:44 smithi153 bash[14340]: cluster 2024-03-12T02:22:43.386479+0000 mgr.smithi153.asomxr (mgr.14195) 432 : cluster 0 pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:45.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:44 smithi163 bash[22347]: cluster 2024-03-12T02:22:43.386479+0000 mgr.smithi153.asomxr (mgr.14195) 432 : cluster 0 pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:45.602 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:22:46.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:45 smithi163 bash[22347]: audit 2024-03-12T02:22:44.735525+0000 mgr.smithi153.asomxr (mgr.14195) 433 : audit 0 from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:46.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:45 smithi163 bash[22347]: audit 2024-03-12T02:22:45.392430+0000 mon.smithi153 (mon.0) 860 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:22:46.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:45 smithi163 bash[22347]: audit 2024-03-12T02:22:45.756162+0000 mon.smithi153 (mon.0) 861 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:22:46.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:45 smithi163 bash[22347]: audit 2024-03-12T02:22:45.764465+0000 mon.smithi153 (mon.0) 862 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:22:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:45 smithi153 bash[14340]: audit 2024-03-12T02:22:44.735525+0000 mgr.smithi153.asomxr (mgr.14195) 433 : audit 0 from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:45 smithi153 bash[14340]: audit 2024-03-12T02:22:45.392430+0000 mon.smithi153 (mon.0) 860 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:22:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:45 smithi153 bash[14340]: audit 2024-03-12T02:22:45.756162+0000 mon.smithi153 (mon.0) 861 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:22:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:45 smithi153 bash[14340]: audit 2024-03-12T02:22:45.764465+0000 mon.smithi153 (mon.0) 862 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:22:46.603 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:22:47.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:46 smithi163 bash[22347]: cluster 2024-03-12T02:22:45.386914+0000 mgr.smithi153.asomxr (mgr.14195) 434 : cluster 0 pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:47.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:46 smithi163 bash[22347]: audit 2024-03-12T02:22:46.236220+0000 mon.smithi153 (mon.0) 863 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:22:47.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:46 smithi163 bash[22347]: audit 2024-03-12T02:22:46.238033+0000 mon.smithi153 (mon.0) 864 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:22:47.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:46 smithi163 bash[22347]: audit 2024-03-12T02:22:46.248167+0000 mon.smithi153 (mon.0) 865 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:22:47.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:46 smithi163 bash[22347]: audit 2024-03-12T02:22:46.252869+0000 mon.smithi153 (mon.0) 866 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:22:47.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:46 smithi153 bash[14340]: cluster 2024-03-12T02:22:45.386914+0000 mgr.smithi153.asomxr (mgr.14195) 434 : cluster 0 pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:47.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:46 smithi153 bash[14340]: audit 2024-03-12T02:22:46.236220+0000 mon.smithi153 (mon.0) 863 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:22:47.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:46 smithi153 bash[14340]: audit 2024-03-12T02:22:46.238033+0000 mon.smithi153 (mon.0) 864 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:22:47.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:46 smithi153 bash[14340]: audit 2024-03-12T02:22:46.248167+0000 mon.smithi153 (mon.0) 865 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:22:47.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:46 smithi153 bash[14340]: audit 2024-03-12T02:22:46.252869+0000 mon.smithi153 (mon.0) 866 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:22:47.984 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:47 smithi163 bash[22347]: cluster 2024-03-12T02:22:46.240676+0000 mgr.smithi153.asomxr (mgr.14195) 435 : cluster 0 pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:47.984 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:47 smithi163 bash[22347]: cluster 2024-03-12T02:22:46.241116+0000 mgr.smithi153.asomxr (mgr.14195) 436 : cluster 0 pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:48.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:47 smithi153 bash[14340]: cluster 2024-03-12T02:22:46.240676+0000 mgr.smithi153.asomxr (mgr.14195) 435 : cluster 0 pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:48.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:47 smithi153 bash[14340]: cluster 2024-03-12T02:22:46.241116+0000 mgr.smithi153.asomxr (mgr.14195) 436 : cluster 0 pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:50.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:49 smithi163 bash[22347]: cluster 2024-03-12T02:22:48.241670+0000 mgr.smithi153.asomxr (mgr.14195) 437 : cluster 0 pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:50.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:49 smithi153 bash[14340]: cluster 2024-03-12T02:22:48.241670+0000 mgr.smithi153.asomxr (mgr.14195) 437 : cluster 0 pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:51.349 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:22:52.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:51 smithi163 bash[22347]: cluster 2024-03-12T02:22:50.242341+0000 mgr.smithi153.asomxr (mgr.14195) 438 : cluster 0 pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:52.274 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:51 smithi153 bash[14340]: cluster 2024-03-12T02:22:50.242341+0000 mgr.smithi153.asomxr (mgr.14195) 438 : cluster 0 pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:54.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:53 smithi163 bash[22347]: cluster 2024-03-12T02:22:52.242812+0000 mgr.smithi153.asomxr (mgr.14195) 439 : cluster 0 pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:54.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:53 smithi153 bash[14340]: cluster 2024-03-12T02:22:52.242812+0000 mgr.smithi153.asomxr (mgr.14195) 439 : cluster 0 pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:54.528 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:22:54.528 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:22:55.370 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:22:56.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:55 smithi163 bash[22347]: cluster 2024-03-12T02:22:54.243373+0000 mgr.smithi153.asomxr (mgr.14195) 440 : cluster 0 pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:56.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:55 smithi163 bash[22347]: audit 2024-03-12T02:22:54.509763+0000 mgr.smithi153.asomxr (mgr.14195) 441 : audit 0 from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:56.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:55 smithi153 bash[14340]: cluster 2024-03-12T02:22:54.243373+0000 mgr.smithi153.asomxr (mgr.14195) 440 : cluster 0 pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:56.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:55 smithi153 bash[14340]: audit 2024-03-12T02:22:54.509763+0000 mgr.smithi153.asomxr (mgr.14195) 441 : audit 0 from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:22:56.373 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:22:58.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:57 smithi163 bash[22347]: cluster 2024-03-12T02:22:56.244023+0000 mgr.smithi153.asomxr (mgr.14195) 442 : cluster 0 pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:22:58.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:57 smithi153 bash[14340]: cluster 2024-03-12T02:22:56.244023+0000 mgr.smithi153.asomxr (mgr.14195) 442 : cluster 0 pgmap v367: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:00.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:22:59 smithi163 bash[22347]: cluster 2024-03-12T02:22:58.244664+0000 mgr.smithi153.asomxr (mgr.14195) 443 : cluster 0 pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:00.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:22:59 smithi153 bash[14340]: cluster 2024-03-12T02:22:58.244664+0000 mgr.smithi153.asomxr (mgr.14195) 443 : cluster 0 pgmap v368: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:01.128 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:23:01.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:00 smithi163 bash[22347]: audit 2024-03-12T02:23:00.393618+0000 mon.smithi153 (mon.0) 867 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:23:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:00 smithi153 bash[14340]: audit 2024-03-12T02:23:00.393618+0000 mon.smithi153 (mon.0) 867 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:23:02.180 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:01 smithi153 bash[14340]: cluster 2024-03-12T02:23:00.245335+0000 mgr.smithi153.asomxr (mgr.14195) 444 : cluster 0 pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:02.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:01 smithi163 bash[22347]: cluster 2024-03-12T02:23:00.245335+0000 mgr.smithi153.asomxr (mgr.14195) 444 : cluster 0 pgmap v369: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:04.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:03 smithi163 bash[22347]: cluster 2024-03-12T02:23:02.245906+0000 mgr.smithi153.asomxr (mgr.14195) 445 : cluster 0 pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:04.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:03 smithi153 bash[14340]: cluster 2024-03-12T02:23:02.245906+0000 mgr.smithi153.asomxr (mgr.14195) 445 : cluster 0 pgmap v370: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:04.549 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:23:04.549 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:23:05.349 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:23:06.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:05 smithi153 bash[14340]: cluster 2024-03-12T02:23:04.246447+0000 mgr.smithi153.asomxr (mgr.14195) 446 : cluster 0 pgmap v371: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:06.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:05 smithi153 bash[14340]: audit 2024-03-12T02:23:04.533937+0000 mgr.smithi153.asomxr (mgr.14195) 447 : audit 0 from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:06.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:05 smithi163 bash[22347]: cluster 2024-03-12T02:23:04.246447+0000 mgr.smithi153.asomxr (mgr.14195) 446 : cluster 0 pgmap v371: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:06.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:05 smithi163 bash[22347]: audit 2024-03-12T02:23:04.533937+0000 mgr.smithi153.asomxr (mgr.14195) 447 : audit 0 from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:06.350 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:23:08.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:07 smithi163 bash[22347]: cluster 2024-03-12T02:23:06.247083+0000 mgr.smithi153.asomxr (mgr.14195) 448 : cluster 0 pgmap v372: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:08.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:07 smithi153 bash[14340]: cluster 2024-03-12T02:23:06.247083+0000 mgr.smithi153.asomxr (mgr.14195) 448 : cluster 0 pgmap v372: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:10.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:09 smithi163 bash[22347]: cluster 2024-03-12T02:23:08.247736+0000 mgr.smithi153.asomxr (mgr.14195) 449 : cluster 0 pgmap v373: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:10.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:09 smithi153 bash[14340]: cluster 2024-03-12T02:23:08.247736+0000 mgr.smithi153.asomxr (mgr.14195) 449 : cluster 0 pgmap v373: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:11.104 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:23:12.212 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:11 smithi153 bash[14340]: cluster 2024-03-12T02:23:10.248360+0000 mgr.smithi153.asomxr (mgr.14195) 450 : cluster 0 pgmap v374: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:12.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:11 smithi163 bash[22347]: cluster 2024-03-12T02:23:10.248360+0000 mgr.smithi153.asomxr (mgr.14195) 450 : cluster 0 pgmap v374: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:14.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:13 smithi153 bash[14340]: cluster 2024-03-12T02:23:12.248824+0000 mgr.smithi153.asomxr (mgr.14195) 451 : cluster 0 pgmap v375: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:14.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:13 smithi163 bash[22347]: cluster 2024-03-12T02:23:12.248824+0000 mgr.smithi153.asomxr (mgr.14195) 451 : cluster 0 pgmap v375: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:14.240 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:23:14.240 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:23:15.020 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:23:16.021 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:23:16.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:15 smithi163 bash[22347]: audit 2024-03-12T02:23:14.228915+0000 mgr.smithi153.asomxr (mgr.14195) 452 : audit 0 from='client.14630 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:16.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:15 smithi163 bash[22347]: cluster 2024-03-12T02:23:14.249303+0000 mgr.smithi153.asomxr (mgr.14195) 453 : cluster 0 pgmap v376: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:16.237 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:15 smithi163 bash[22347]: audit 2024-03-12T02:23:15.394156+0000 mon.smithi153 (mon.0) 868 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:23:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:15 smithi153 bash[14340]: audit 2024-03-12T02:23:14.228915+0000 mgr.smithi153.asomxr (mgr.14195) 452 : audit 0 from='client.14630 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:15 smithi153 bash[14340]: cluster 2024-03-12T02:23:14.249303+0000 mgr.smithi153.asomxr (mgr.14195) 453 : cluster 0 pgmap v376: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:15 smithi153 bash[14340]: audit 2024-03-12T02:23:15.394156+0000 mon.smithi153 (mon.0) 868 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:23:18.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:17 smithi163 bash[22347]: cluster 2024-03-12T02:23:16.249967+0000 mgr.smithi153.asomxr (mgr.14195) 454 : cluster 0 pgmap v377: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:18.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:17 smithi153 bash[14340]: cluster 2024-03-12T02:23:16.249967+0000 mgr.smithi153.asomxr (mgr.14195) 454 : cluster 0 pgmap v377: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:20.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:19 smithi163 bash[22347]: cluster 2024-03-12T02:23:18.250665+0000 mgr.smithi153.asomxr (mgr.14195) 455 : cluster 0 pgmap v378: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:20.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:19 smithi153 bash[14340]: cluster 2024-03-12T02:23:18.250665+0000 mgr.smithi153.asomxr (mgr.14195) 455 : cluster 0 pgmap v378: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:20.771 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:23:22.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:21 smithi153 bash[14340]: cluster 2024-03-12T02:23:20.251256+0000 mgr.smithi153.asomxr (mgr.14195) 456 : cluster 0 pgmap v379: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:22.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:21 smithi163 bash[22347]: cluster 2024-03-12T02:23:20.251256+0000 mgr.smithi153.asomxr (mgr.14195) 456 : cluster 0 pgmap v379: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:24.035 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:23:24.035 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:23:24.227 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:23 smithi153 bash[14340]: cluster 2024-03-12T02:23:22.251871+0000 mgr.smithi153.asomxr (mgr.14195) 457 : cluster 0 pgmap v380: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:24.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:23 smithi163 bash[22347]: cluster 2024-03-12T02:23:22.251871+0000 mgr.smithi153.asomxr (mgr.14195) 457 : cluster 0 pgmap v380: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:24.788 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:23:25.789 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:23:26.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:25 smithi153 bash[14340]: audit 2024-03-12T02:23:24.019891+0000 mgr.smithi153.asomxr (mgr.14195) 458 : audit 0 from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:26.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:25 smithi153 bash[14340]: cluster 2024-03-12T02:23:24.252619+0000 mgr.smithi153.asomxr (mgr.14195) 459 : cluster 0 pgmap v381: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:26.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:25 smithi163 bash[22347]: audit 2024-03-12T02:23:24.019891+0000 mgr.smithi153.asomxr (mgr.14195) 458 : audit 0 from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:26.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:25 smithi163 bash[22347]: cluster 2024-03-12T02:23:24.252619+0000 mgr.smithi153.asomxr (mgr.14195) 459 : cluster 0 pgmap v381: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:28.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:27 smithi163 bash[22347]: cluster 2024-03-12T02:23:26.253318+0000 mgr.smithi153.asomxr (mgr.14195) 460 : cluster 0 pgmap v382: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:28.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:27 smithi153 bash[14340]: cluster 2024-03-12T02:23:26.253318+0000 mgr.smithi153.asomxr (mgr.14195) 460 : cluster 0 pgmap v382: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:30.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:29 smithi163 bash[22347]: cluster 2024-03-12T02:23:28.253989+0000 mgr.smithi153.asomxr (mgr.14195) 461 : cluster 0 pgmap v383: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:30.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:29 smithi153 bash[14340]: cluster 2024-03-12T02:23:28.253989+0000 mgr.smithi153.asomxr (mgr.14195) 461 : cluster 0 pgmap v383: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:30.543 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:23:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:30 smithi153 bash[14340]: audit 2024-03-12T02:23:30.393488+0000 mon.smithi153 (mon.0) 869 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:23:31.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:30 smithi163 bash[22347]: audit 2024-03-12T02:23:30.393488+0000 mon.smithi153 (mon.0) 869 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:23:32.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:31 smithi153 bash[14340]: cluster 2024-03-12T02:23:30.254659+0000 mgr.smithi153.asomxr (mgr.14195) 462 : cluster 0 pgmap v384: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:32.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:31 smithi163 bash[22347]: cluster 2024-03-12T02:23:30.254659+0000 mgr.smithi153.asomxr (mgr.14195) 462 : cluster 0 pgmap v384: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:33.800 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:23:33.800 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:23:34.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:33 smithi153 bash[14340]: cluster 2024-03-12T02:23:32.255280+0000 mgr.smithi153.asomxr (mgr.14195) 463 : cluster 0 pgmap v385: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:34.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:33 smithi163 bash[22347]: cluster 2024-03-12T02:23:32.255280+0000 mgr.smithi153.asomxr (mgr.14195) 463 : cluster 0 pgmap v385: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:34.579 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:23:35.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:34 smithi153 bash[14340]: audit 2024-03-12T02:23:33.788846+0000 mgr.smithi153.asomxr (mgr.14195) 464 : audit 0 from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:35.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:35 smithi163 bash[22347]: audit 2024-03-12T02:23:33.788846+0000 mgr.smithi153.asomxr (mgr.14195) 464 : audit 0 from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:35.581 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:23:36.314 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:35 smithi153 bash[14340]: cluster 2024-03-12T02:23:34.255926+0000 mgr.smithi153.asomxr (mgr.14195) 465 : cluster 0 pgmap v386: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:36.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:36 smithi163 bash[22347]: cluster 2024-03-12T02:23:34.255926+0000 mgr.smithi153.asomxr (mgr.14195) 465 : cluster 0 pgmap v386: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:37.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:37 smithi153 bash[14340]: cluster 2024-03-12T02:23:36.256555+0000 mgr.smithi153.asomxr (mgr.14195) 466 : cluster 0 pgmap v387: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:37.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:37 smithi163 bash[22347]: cluster 2024-03-12T02:23:36.256555+0000 mgr.smithi153.asomxr (mgr.14195) 466 : cluster 0 pgmap v387: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:40.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:39 smithi153 bash[14340]: cluster 2024-03-12T02:23:38.257135+0000 mgr.smithi153.asomxr (mgr.14195) 467 : cluster 0 pgmap v388: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:40.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:39 smithi163 bash[22347]: cluster 2024-03-12T02:23:38.257135+0000 mgr.smithi153.asomxr (mgr.14195) 467 : cluster 0 pgmap v388: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:40.335 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:23:42.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:41 smithi153 bash[14340]: cluster 2024-03-12T02:23:40.257765+0000 mgr.smithi153.asomxr (mgr.14195) 468 : cluster 0 pgmap v389: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:42.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:41 smithi163 bash[22347]: cluster 2024-03-12T02:23:40.257765+0000 mgr.smithi153.asomxr (mgr.14195) 468 : cluster 0 pgmap v389: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:43.539 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:23:43.540 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:23:43.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:43 smithi153 bash[14340]: cluster 2024-03-12T02:23:42.258364+0000 mgr.smithi153.asomxr (mgr.14195) 469 : cluster 0 pgmap v390: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:44.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:43 smithi163 bash[22347]: cluster 2024-03-12T02:23:42.258364+0000 mgr.smithi153.asomxr (mgr.14195) 469 : cluster 0 pgmap v390: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:44.404 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:23:45.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:44 smithi153 bash[14340]: audit 2024-03-12T02:23:43.523381+0000 mgr.smithi153.asomxr (mgr.14195) 470 : audit 0 from='client.14642 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:45.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:44 smithi163 bash[22347]: audit 2024-03-12T02:23:43.523381+0000 mgr.smithi153.asomxr (mgr.14195) 470 : audit 0 from='client.14642 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:45.405 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:23:46.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:45 smithi153 bash[14340]: cluster 2024-03-12T02:23:44.258994+0000 mgr.smithi153.asomxr (mgr.14195) 471 : cluster 0 pgmap v391: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:46.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:45 smithi153 bash[14340]: audit 2024-03-12T02:23:45.394743+0000 mon.smithi153 (mon.0) 870 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:23:46.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:45 smithi163 bash[22347]: cluster 2024-03-12T02:23:44.258994+0000 mgr.smithi153.asomxr (mgr.14195) 471 : cluster 0 pgmap v391: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:46.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:45 smithi163 bash[22347]: audit 2024-03-12T02:23:45.394743+0000 mon.smithi153 (mon.0) 870 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:23:47.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:46 smithi153 bash[14340]: audit 2024-03-12T02:23:46.341987+0000 mon.smithi153 (mon.0) 871 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:23:47.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:46 smithi163 bash[22347]: audit 2024-03-12T02:23:46.341987+0000 mon.smithi153 (mon.0) 871 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:23:48.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:47 smithi153 bash[14340]: cluster 2024-03-12T02:23:46.259626+0000 mgr.smithi153.asomxr (mgr.14195) 472 : cluster 0 pgmap v392: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:48.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:47 smithi153 bash[14340]: audit 2024-03-12T02:23:46.925879+0000 mon.smithi153 (mon.0) 872 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:23:48.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:47 smithi153 bash[14340]: audit 2024-03-12T02:23:46.927827+0000 mon.smithi153 (mon.0) 873 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:23:48.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:47 smithi153 bash[14340]: cluster 2024-03-12T02:23:46.930586+0000 mgr.smithi153.asomxr (mgr.14195) 473 : cluster 0 pgmap v393: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:48.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:47 smithi153 bash[14340]: cluster 2024-03-12T02:23:46.930929+0000 mgr.smithi153.asomxr (mgr.14195) 474 : cluster 0 pgmap v394: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:48.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:47 smithi153 bash[14340]: audit 2024-03-12T02:23:46.937907+0000 mon.smithi153 (mon.0) 874 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:23:48.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:47 smithi153 bash[14340]: audit 2024-03-12T02:23:46.942349+0000 mon.smithi153 (mon.0) 875 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:23:48.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:47 smithi163 bash[22347]: cluster 2024-03-12T02:23:46.259626+0000 mgr.smithi153.asomxr (mgr.14195) 472 : cluster 0 pgmap v392: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:48.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:47 smithi163 bash[22347]: audit 2024-03-12T02:23:46.925879+0000 mon.smithi153 (mon.0) 872 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:23:48.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:47 smithi163 bash[22347]: audit 2024-03-12T02:23:46.927827+0000 mon.smithi153 (mon.0) 873 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:23:48.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:47 smithi163 bash[22347]: cluster 2024-03-12T02:23:46.930586+0000 mgr.smithi153.asomxr (mgr.14195) 473 : cluster 0 pgmap v393: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:48.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:47 smithi163 bash[22347]: cluster 2024-03-12T02:23:46.930929+0000 mgr.smithi153.asomxr (mgr.14195) 474 : cluster 0 pgmap v394: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:48.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:47 smithi163 bash[22347]: audit 2024-03-12T02:23:46.937907+0000 mon.smithi153 (mon.0) 874 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:23:48.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:47 smithi163 bash[22347]: audit 2024-03-12T02:23:46.942349+0000 mon.smithi153 (mon.0) 875 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:23:50.145 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:23:50.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:49 smithi153 bash[14340]: cluster 2024-03-12T02:23:48.931480+0000 mgr.smithi153.asomxr (mgr.14195) 475 : cluster 0 pgmap v395: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:50.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:49 smithi163 bash[22347]: cluster 2024-03-12T02:23:48.931480+0000 mgr.smithi153.asomxr (mgr.14195) 475 : cluster 0 pgmap v395: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:52.265 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:51 smithi153 bash[14340]: cluster 2024-03-12T02:23:50.932027+0000 mgr.smithi153.asomxr (mgr.14195) 476 : cluster 0 pgmap v396: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:52.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:51 smithi163 bash[22347]: cluster 2024-03-12T02:23:50.932027+0000 mgr.smithi153.asomxr (mgr.14195) 476 : cluster 0 pgmap v396: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:53.557 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:23:53.557 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:23:54.067 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:53 smithi163 bash[22347]: cluster 2024-03-12T02:23:52.932541+0000 mgr.smithi153.asomxr (mgr.14195) 477 : cluster 0 pgmap v397: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:54.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:53 smithi153 bash[14340]: cluster 2024-03-12T02:23:52.932541+0000 mgr.smithi153.asomxr (mgr.14195) 477 : cluster 0 pgmap v397: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:54.428 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:23:55.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:54 smithi153 bash[14340]: audit 2024-03-12T02:23:53.540969+0000 mgr.smithi153.asomxr (mgr.14195) 478 : audit 0 from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:55.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:54 smithi163 bash[22347]: audit 2024-03-12T02:23:53.540969+0000 mgr.smithi153.asomxr (mgr.14195) 478 : audit 0 from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:23:55.429 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:23:56.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:55 smithi153 bash[14340]: cluster 2024-03-12T02:23:54.933113+0000 mgr.smithi153.asomxr (mgr.14195) 479 : cluster 0 pgmap v398: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:56.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:55 smithi163 bash[22347]: cluster 2024-03-12T02:23:54.933113+0000 mgr.smithi153.asomxr (mgr.14195) 479 : cluster 0 pgmap v398: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:58.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:57 smithi153 bash[14340]: cluster 2024-03-12T02:23:56.933749+0000 mgr.smithi153.asomxr (mgr.14195) 480 : cluster 0 pgmap v399: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:23:58.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:57 smithi163 bash[22347]: cluster 2024-03-12T02:23:56.933749+0000 mgr.smithi153.asomxr (mgr.14195) 480 : cluster 0 pgmap v399: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:00.181 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:24:00.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:23:59 smithi153 bash[14340]: cluster 2024-03-12T02:23:58.934385+0000 mgr.smithi153.asomxr (mgr.14195) 481 : cluster 0 pgmap v400: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:00.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:23:59 smithi163 bash[22347]: cluster 2024-03-12T02:23:58.934385+0000 mgr.smithi153.asomxr (mgr.14195) 481 : cluster 0 pgmap v400: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:01 smithi153 bash[14340]: audit 2024-03-12T02:24:00.394471+0000 mon.smithi153 (mon.0) 876 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:24:01.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:01 smithi163 bash[22347]: audit 2024-03-12T02:24:00.394471+0000 mon.smithi153 (mon.0) 876 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:24:02.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:02 smithi153 bash[14340]: cluster 2024-03-12T02:24:00.935024+0000 mgr.smithi153.asomxr (mgr.14195) 482 : cluster 0 pgmap v401: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:02.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:02 smithi163 bash[22347]: cluster 2024-03-12T02:24:00.935024+0000 mgr.smithi153.asomxr (mgr.14195) 482 : cluster 0 pgmap v401: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:03.620 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:24:03.620 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:24:04.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:03 smithi153 bash[14340]: cluster 2024-03-12T02:24:02.935536+0000 mgr.smithi153.asomxr (mgr.14195) 483 : cluster 0 pgmap v402: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:04.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:03 smithi163 bash[22347]: cluster 2024-03-12T02:24:02.935536+0000 mgr.smithi153.asomxr (mgr.14195) 483 : cluster 0 pgmap v402: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:04.439 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:24:05.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:04 smithi153 bash[14340]: audit 2024-03-12T02:24:03.602861+0000 mgr.smithi153.asomxr (mgr.14195) 484 : audit 0 from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:05.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:04 smithi163 bash[22347]: audit 2024-03-12T02:24:03.602861+0000 mgr.smithi153.asomxr (mgr.14195) 484 : audit 0 from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:05.440 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:24:06.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:05 smithi153 bash[14340]: cluster 2024-03-12T02:24:04.936146+0000 mgr.smithi153.asomxr (mgr.14195) 485 : cluster 0 pgmap v403: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:06.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:05 smithi163 bash[22347]: cluster 2024-03-12T02:24:04.936146+0000 mgr.smithi153.asomxr (mgr.14195) 485 : cluster 0 pgmap v403: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:08.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:07 smithi153 bash[14340]: cluster 2024-03-12T02:24:06.936671+0000 mgr.smithi153.asomxr (mgr.14195) 486 : cluster 0 pgmap v404: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:08.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:07 smithi163 bash[22347]: cluster 2024-03-12T02:24:06.936671+0000 mgr.smithi153.asomxr (mgr.14195) 486 : cluster 0 pgmap v404: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:10.194 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:24:10.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:09 smithi153 bash[14340]: cluster 2024-03-12T02:24:08.937314+0000 mgr.smithi153.asomxr (mgr.14195) 487 : cluster 0 pgmap v405: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:10.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:10 smithi163 bash[22347]: cluster 2024-03-12T02:24:08.937314+0000 mgr.smithi153.asomxr (mgr.14195) 487 : cluster 0 pgmap v405: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:12.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:12 smithi153 bash[14340]: cluster 2024-03-12T02:24:10.937925+0000 mgr.smithi153.asomxr (mgr.14195) 488 : cluster 0 pgmap v406: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:12.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:12 smithi163 bash[22347]: cluster 2024-03-12T02:24:10.937925+0000 mgr.smithi153.asomxr (mgr.14195) 488 : cluster 0 pgmap v406: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:13.523 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:24:13.523 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:24:14.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:13 smithi153 bash[14340]: cluster 2024-03-12T02:24:12.938584+0000 mgr.smithi153.asomxr (mgr.14195) 489 : cluster 0 pgmap v407: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:14.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:13 smithi163 bash[22347]: cluster 2024-03-12T02:24:12.938584+0000 mgr.smithi153.asomxr (mgr.14195) 489 : cluster 0 pgmap v407: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:14.354 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:24:15.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:14 smithi153 bash[14340]: audit 2024-03-12T02:24:13.508162+0000 mgr.smithi153.asomxr (mgr.14195) 490 : audit 0 from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:15.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:14 smithi163 bash[22347]: audit 2024-03-12T02:24:13.508162+0000 mgr.smithi153.asomxr (mgr.14195) 490 : audit 0 from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:15.355 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:24:16.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:15 smithi153 bash[14340]: cluster 2024-03-12T02:24:14.939230+0000 mgr.smithi153.asomxr (mgr.14195) 491 : cluster 0 pgmap v408: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:16.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:15 smithi153 bash[14340]: audit 2024-03-12T02:24:15.395180+0000 mon.smithi153 (mon.0) 877 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:24:16.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:15 smithi163 bash[22347]: cluster 2024-03-12T02:24:14.939230+0000 mgr.smithi153.asomxr (mgr.14195) 491 : cluster 0 pgmap v408: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:16.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:15 smithi163 bash[22347]: audit 2024-03-12T02:24:15.395180+0000 mon.smithi153 (mon.0) 877 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:24:18.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:17 smithi153 bash[14340]: cluster 2024-03-12T02:24:16.939908+0000 mgr.smithi153.asomxr (mgr.14195) 492 : cluster 0 pgmap v409: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:18.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:17 smithi163 bash[22347]: cluster 2024-03-12T02:24:16.939908+0000 mgr.smithi153.asomxr (mgr.14195) 492 : cluster 0 pgmap v409: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:20.109 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:24:20.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:20 smithi153 bash[14340]: cluster 2024-03-12T02:24:18.940542+0000 mgr.smithi153.asomxr (mgr.14195) 493 : cluster 0 pgmap v410: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:20.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:20 smithi163 bash[22347]: cluster 2024-03-12T02:24:18.940542+0000 mgr.smithi153.asomxr (mgr.14195) 493 : cluster 0 pgmap v410: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:22.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:22 smithi153 bash[14340]: cluster 2024-03-12T02:24:20.941121+0000 mgr.smithi153.asomxr (mgr.14195) 494 : cluster 0 pgmap v411: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:22.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:22 smithi163 bash[22347]: cluster 2024-03-12T02:24:20.941121+0000 mgr.smithi153.asomxr (mgr.14195) 494 : cluster 0 pgmap v411: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:23.589 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:24:23.589 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:24:24.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:23 smithi153 bash[14340]: cluster 2024-03-12T02:24:22.941666+0000 mgr.smithi153.asomxr (mgr.14195) 495 : cluster 0 pgmap v412: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:24.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:23 smithi163 bash[22347]: cluster 2024-03-12T02:24:22.941666+0000 mgr.smithi153.asomxr (mgr.14195) 495 : cluster 0 pgmap v412: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:24.390 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:24:24.984 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:24 smithi163 bash[22347]: audit 2024-03-12T02:24:23.573409+0000 mgr.smithi153.asomxr (mgr.14195) 496 : audit 0 from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:25.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:24 smithi153 bash[14340]: audit 2024-03-12T02:24:23.573409+0000 mgr.smithi153.asomxr (mgr.14195) 496 : audit 0 from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:25.391 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:24:26.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:25 smithi153 bash[14340]: cluster 2024-03-12T02:24:24.942303+0000 mgr.smithi153.asomxr (mgr.14195) 497 : cluster 0 pgmap v413: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:26.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:25 smithi163 bash[22347]: cluster 2024-03-12T02:24:24.942303+0000 mgr.smithi153.asomxr (mgr.14195) 497 : cluster 0 pgmap v413: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:28.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:28 smithi153 bash[14340]: cluster 2024-03-12T02:24:26.942918+0000 mgr.smithi153.asomxr (mgr.14195) 498 : cluster 0 pgmap v414: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:28.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:28 smithi163 bash[22347]: cluster 2024-03-12T02:24:26.942918+0000 mgr.smithi153.asomxr (mgr.14195) 498 : cluster 0 pgmap v414: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:30.143 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:24:30.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:30 smithi153 bash[14340]: cluster 2024-03-12T02:24:28.943580+0000 mgr.smithi153.asomxr (mgr.14195) 499 : cluster 0 pgmap v415: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:30.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:30 smithi163 bash[22347]: cluster 2024-03-12T02:24:28.943580+0000 mgr.smithi153.asomxr (mgr.14195) 499 : cluster 0 pgmap v415: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:31 smithi153 bash[14340]: audit 2024-03-12T02:24:30.395195+0000 mon.smithi153 (mon.0) 878 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:24:31.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:31 smithi163 bash[22347]: audit 2024-03-12T02:24:30.395195+0000 mon.smithi153 (mon.0) 878 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:24:32.310 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:32 smithi153 bash[14340]: cluster 2024-03-12T02:24:30.944162+0000 mgr.smithi153.asomxr (mgr.14195) 500 : cluster 0 pgmap v416: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:32.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:32 smithi163 bash[22347]: cluster 2024-03-12T02:24:30.944162+0000 mgr.smithi153.asomxr (mgr.14195) 500 : cluster 0 pgmap v416: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:33.695 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:24:33.695 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:24:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:33 smithi153 bash[14340]: cluster 2024-03-12T02:24:32.944571+0000 mgr.smithi153.asomxr (mgr.14195) 501 : cluster 0 pgmap v417: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:34.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:33 smithi163 bash[22347]: cluster 2024-03-12T02:24:32.944571+0000 mgr.smithi153.asomxr (mgr.14195) 501 : cluster 0 pgmap v417: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:34.500 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:24:35.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:34 smithi153 bash[14340]: audit 2024-03-12T02:24:33.679293+0000 mgr.smithi153.asomxr (mgr.14195) 502 : audit 0 from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:35.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:34 smithi163 bash[22347]: audit 2024-03-12T02:24:33.679293+0000 mgr.smithi153.asomxr (mgr.14195) 502 : audit 0 from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:35.502 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:24:36.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:35 smithi153 bash[14340]: cluster 2024-03-12T02:24:34.945137+0000 mgr.smithi153.asomxr (mgr.14195) 503 : cluster 0 pgmap v418: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:36.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:35 smithi163 bash[22347]: cluster 2024-03-12T02:24:34.945137+0000 mgr.smithi153.asomxr (mgr.14195) 503 : cluster 0 pgmap v418: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:38.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:38 smithi153 bash[14340]: cluster 2024-03-12T02:24:36.945786+0000 mgr.smithi153.asomxr (mgr.14195) 504 : cluster 0 pgmap v419: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:38.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:38 smithi163 bash[22347]: cluster 2024-03-12T02:24:36.945786+0000 mgr.smithi153.asomxr (mgr.14195) 504 : cluster 0 pgmap v419: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:40.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:40 smithi163 bash[22347]: cluster 2024-03-12T02:24:38.946544+0000 mgr.smithi153.asomxr (mgr.14195) 505 : cluster 0 pgmap v420: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:40.250 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:24:40.279 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:40 smithi153 bash[14340]: cluster 2024-03-12T02:24:38.946544+0000 mgr.smithi153.asomxr (mgr.14195) 505 : cluster 0 pgmap v420: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:41.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:41 smithi153 bash[14340]: cluster 2024-03-12T02:24:40.947272+0000 mgr.smithi153.asomxr (mgr.14195) 506 : cluster 0 pgmap v421: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:41.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:41 smithi163 bash[22347]: cluster 2024-03-12T02:24:40.947272+0000 mgr.smithi153.asomxr (mgr.14195) 506 : cluster 0 pgmap v421: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:43.429 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:24:43.429 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:24:44.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:43 smithi153 bash[14340]: cluster 2024-03-12T02:24:42.947990+0000 mgr.smithi153.asomxr (mgr.14195) 507 : cluster 0 pgmap v422: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:44.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:43 smithi163 bash[22347]: cluster 2024-03-12T02:24:42.947990+0000 mgr.smithi153.asomxr (mgr.14195) 507 : cluster 0 pgmap v422: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:44.277 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:24:45.052 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:44 smithi163 bash[22347]: audit 2024-03-12T02:24:43.411114+0000 mgr.smithi153.asomxr (mgr.14195) 508 : audit 0 from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:45.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:44 smithi153 bash[14340]: audit 2024-03-12T02:24:43.411114+0000 mgr.smithi153.asomxr (mgr.14195) 508 : audit 0 from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:45.278 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:24:46.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:45 smithi153 bash[14340]: cluster 2024-03-12T02:24:44.948674+0000 mgr.smithi153.asomxr (mgr.14195) 509 : cluster 0 pgmap v423: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:46.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:45 smithi153 bash[14340]: audit 2024-03-12T02:24:45.395690+0000 mon.smithi153 (mon.0) 879 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:24:46.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:45 smithi163 bash[22347]: cluster 2024-03-12T02:24:44.948674+0000 mgr.smithi153.asomxr (mgr.14195) 509 : cluster 0 pgmap v423: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:46.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:45 smithi163 bash[22347]: audit 2024-03-12T02:24:45.395690+0000 mon.smithi153 (mon.0) 879 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:24:48.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:48 smithi153 bash[14340]: cluster 2024-03-12T02:24:46.949265+0000 mgr.smithi153.asomxr (mgr.14195) 510 : cluster 0 pgmap v424: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:48.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:48 smithi153 bash[14340]: audit 2024-03-12T02:24:47.036149+0000 mon.smithi153 (mon.0) 880 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:24:48.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:48 smithi153 bash[14340]: audit 2024-03-12T02:24:47.612764+0000 mon.smithi153 (mon.0) 881 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:24:48.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:48 smithi153 bash[14340]: audit 2024-03-12T02:24:47.614437+0000 mon.smithi153 (mon.0) 882 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:24:48.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:48 smithi153 bash[14340]: audit 2024-03-12T02:24:47.624439+0000 mon.smithi153 (mon.0) 883 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:24:48.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:48 smithi153 bash[14340]: audit 2024-03-12T02:24:47.629101+0000 mon.smithi153 (mon.0) 884 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:24:48.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:48 smithi163 bash[22347]: cluster 2024-03-12T02:24:46.949265+0000 mgr.smithi153.asomxr (mgr.14195) 510 : cluster 0 pgmap v424: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:48.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:48 smithi163 bash[22347]: audit 2024-03-12T02:24:47.036149+0000 mon.smithi153 (mon.0) 880 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:24:48.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:48 smithi163 bash[22347]: audit 2024-03-12T02:24:47.612764+0000 mon.smithi153 (mon.0) 881 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:24:48.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:48 smithi163 bash[22347]: audit 2024-03-12T02:24:47.614437+0000 mon.smithi153 (mon.0) 882 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:24:48.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:48 smithi163 bash[22347]: audit 2024-03-12T02:24:47.624439+0000 mon.smithi153 (mon.0) 883 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:24:48.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:48 smithi163 bash[22347]: audit 2024-03-12T02:24:47.629101+0000 mon.smithi153 (mon.0) 884 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:24:49.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:49 smithi153 bash[14340]: cluster 2024-03-12T02:24:47.617218+0000 mgr.smithi153.asomxr (mgr.14195) 511 : cluster 0 pgmap v425: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:49.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:49 smithi153 bash[14340]: cluster 2024-03-12T02:24:47.617700+0000 mgr.smithi153.asomxr (mgr.14195) 512 : cluster 0 pgmap v426: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:49.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:49 smithi163 bash[22347]: cluster 2024-03-12T02:24:47.617218+0000 mgr.smithi153.asomxr (mgr.14195) 511 : cluster 0 pgmap v425: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:49.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:49 smithi163 bash[22347]: cluster 2024-03-12T02:24:47.617700+0000 mgr.smithi153.asomxr (mgr.14195) 512 : cluster 0 pgmap v426: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:50.027 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:24:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:50 smithi153 bash[14340]: cluster 2024-03-12T02:24:49.618333+0000 mgr.smithi153.asomxr (mgr.14195) 513 : cluster 0 pgmap v427: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:50.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:50 smithi163 bash[22347]: cluster 2024-03-12T02:24:49.618333+0000 mgr.smithi153.asomxr (mgr.14195) 513 : cluster 0 pgmap v427: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:52.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:52 smithi153 bash[14340]: cluster 2024-03-12T02:24:51.618978+0000 mgr.smithi153.asomxr (mgr.14195) 514 : cluster 0 pgmap v428: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:52.984 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:52 smithi163 bash[22347]: cluster 2024-03-12T02:24:51.618978+0000 mgr.smithi153.asomxr (mgr.14195) 514 : cluster 0 pgmap v428: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:53.297 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:24:53.297 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:24:54.094 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:24:55.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:54 smithi153 bash[14340]: audit 2024-03-12T02:24:53.280291+0000 mgr.smithi153.asomxr (mgr.14195) 515 : audit 0 from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:55.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:54 smithi153 bash[14340]: cluster 2024-03-12T02:24:53.619623+0000 mgr.smithi153.asomxr (mgr.14195) 516 : cluster 0 pgmap v429: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:55.095 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:24:55.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:54 smithi163 bash[22347]: audit 2024-03-12T02:24:53.280291+0000 mgr.smithi153.asomxr (mgr.14195) 515 : audit 0 from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:24:55.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:54 smithi163 bash[22347]: cluster 2024-03-12T02:24:53.619623+0000 mgr.smithi153.asomxr (mgr.14195) 516 : cluster 0 pgmap v429: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:57.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:56 smithi153 bash[14340]: cluster 2024-03-12T02:24:55.620261+0000 mgr.smithi153.asomxr (mgr.14195) 517 : cluster 0 pgmap v430: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:57.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:56 smithi163 bash[22347]: cluster 2024-03-12T02:24:55.620261+0000 mgr.smithi153.asomxr (mgr.14195) 517 : cluster 0 pgmap v430: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:59.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:24:58 smithi153 bash[14340]: cluster 2024-03-12T02:24:57.620912+0000 mgr.smithi153.asomxr (mgr.14195) 518 : cluster 0 pgmap v431: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:59.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:24:58 smithi163 bash[22347]: cluster 2024-03-12T02:24:57.620912+0000 mgr.smithi153.asomxr (mgr.14195) 518 : cluster 0 pgmap v431: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:24:59.844 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:25:01.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:00 smithi153 bash[14340]: cluster 2024-03-12T02:24:59.621685+0000 mgr.smithi153.asomxr (mgr.14195) 519 : cluster 0 pgmap v432: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:01.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:00 smithi153 bash[14340]: audit 2024-03-12T02:25:00.396534+0000 mon.smithi153 (mon.0) 885 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:25:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:00 smithi163 bash[22347]: cluster 2024-03-12T02:24:59.621685+0000 mgr.smithi153.asomxr (mgr.14195) 519 : cluster 0 pgmap v432: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:01.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:00 smithi163 bash[22347]: audit 2024-03-12T02:25:00.396534+0000 mon.smithi153 (mon.0) 885 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:25:02.900 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:02 smithi153 bash[14340]: cluster 2024-03-12T02:25:01.622313+0000 mgr.smithi153.asomxr (mgr.14195) 520 : cluster 0 pgmap v433: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:03.135 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:25:03.135 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:25:03.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:02 smithi163 bash[22347]: cluster 2024-03-12T02:25:01.622313+0000 mgr.smithi153.asomxr (mgr.14195) 520 : cluster 0 pgmap v433: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:03.928 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:25:04.929 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:25:05.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:04 smithi153 bash[14340]: audit 2024-03-12T02:25:03.119056+0000 mgr.smithi153.asomxr (mgr.14195) 521 : audit 0 from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:05.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:04 smithi153 bash[14340]: cluster 2024-03-12T02:25:03.623044+0000 mgr.smithi153.asomxr (mgr.14195) 522 : cluster 0 pgmap v434: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:05.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:04 smithi163 bash[22347]: audit 2024-03-12T02:25:03.119056+0000 mgr.smithi153.asomxr (mgr.14195) 521 : audit 0 from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:05.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:04 smithi163 bash[22347]: cluster 2024-03-12T02:25:03.623044+0000 mgr.smithi153.asomxr (mgr.14195) 522 : cluster 0 pgmap v434: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:07.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:06 smithi153 bash[14340]: cluster 2024-03-12T02:25:05.623739+0000 mgr.smithi153.asomxr (mgr.14195) 523 : cluster 0 pgmap v435: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:07.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:06 smithi163 bash[22347]: cluster 2024-03-12T02:25:05.623739+0000 mgr.smithi153.asomxr (mgr.14195) 523 : cluster 0 pgmap v435: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:09.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:08 smithi153 bash[14340]: cluster 2024-03-12T02:25:07.624376+0000 mgr.smithi153.asomxr (mgr.14195) 524 : cluster 0 pgmap v436: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:09.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:08 smithi163 bash[22347]: cluster 2024-03-12T02:25:07.624376+0000 mgr.smithi153.asomxr (mgr.14195) 524 : cluster 0 pgmap v436: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:09.681 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:25:11.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:10 smithi153 bash[14340]: cluster 2024-03-12T02:25:09.625090+0000 mgr.smithi153.asomxr (mgr.14195) 525 : cluster 0 pgmap v437: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:11.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:10 smithi163 bash[22347]: cluster 2024-03-12T02:25:09.625090+0000 mgr.smithi153.asomxr (mgr.14195) 525 : cluster 0 pgmap v437: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:12.996 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:25:12.996 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:25:13.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:12 smithi153 bash[14340]: cluster 2024-03-12T02:25:11.625760+0000 mgr.smithi153.asomxr (mgr.14195) 526 : cluster 0 pgmap v438: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:13.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:12 smithi163 bash[22347]: cluster 2024-03-12T02:25:11.625760+0000 mgr.smithi153.asomxr (mgr.14195) 526 : cluster 0 pgmap v438: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:13.785 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:25:14.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:13 smithi153 bash[14340]: audit 2024-03-12T02:25:12.980172+0000 mgr.smithi153.asomxr (mgr.14195) 527 : audit 0 from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:14.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:13 smithi163 bash[22347]: audit 2024-03-12T02:25:12.980172+0000 mgr.smithi153.asomxr (mgr.14195) 527 : audit 0 from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:14.786 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:25:15.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:14 smithi153 bash[14340]: cluster 2024-03-12T02:25:13.626394+0000 mgr.smithi153.asomxr (mgr.14195) 528 : cluster 0 pgmap v439: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:15.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:14 smithi163 bash[22347]: cluster 2024-03-12T02:25:13.626394+0000 mgr.smithi153.asomxr (mgr.14195) 528 : cluster 0 pgmap v439: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:15.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:15 smithi163 bash[22347]: audit 2024-03-12T02:25:15.396928+0000 mon.smithi153 (mon.0) 886 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:25:16.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:15 smithi153 bash[14340]: audit 2024-03-12T02:25:15.396928+0000 mon.smithi153 (mon.0) 886 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:25:17.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:16 smithi163 bash[22347]: cluster 2024-03-12T02:25:15.706377+0000 mgr.smithi153.asomxr (mgr.14195) 529 : cluster 0 pgmap v440: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:17.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:16 smithi153 bash[14340]: cluster 2024-03-12T02:25:15.706377+0000 mgr.smithi153.asomxr (mgr.14195) 529 : cluster 0 pgmap v440: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:19.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:18 smithi163 bash[22347]: cluster 2024-03-12T02:25:17.707116+0000 mgr.smithi153.asomxr (mgr.14195) 530 : cluster 0 pgmap v441: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:19.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:18 smithi153 bash[14340]: cluster 2024-03-12T02:25:17.707116+0000 mgr.smithi153.asomxr (mgr.14195) 530 : cluster 0 pgmap v441: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:19.537 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:25:20.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:20 smithi163 bash[22347]: cluster 2024-03-12T02:25:19.707899+0000 mgr.smithi153.asomxr (mgr.14195) 531 : cluster 0 pgmap v442: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:21.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:20 smithi153 bash[14340]: cluster 2024-03-12T02:25:19.707899+0000 mgr.smithi153.asomxr (mgr.14195) 531 : cluster 0 pgmap v442: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:22.748 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:25:22.748 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:25:23.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:22 smithi153 bash[14340]: cluster 2024-03-12T02:25:21.708686+0000 mgr.smithi153.asomxr (mgr.14195) 532 : cluster 0 pgmap v443: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:23.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:22 smithi163 bash[22347]: cluster 2024-03-12T02:25:21.708686+0000 mgr.smithi153.asomxr (mgr.14195) 532 : cluster 0 pgmap v443: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:23.536 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:25:24.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:23 smithi163 bash[22347]: audit 2024-03-12T02:25:22.731847+0000 mgr.smithi153.asomxr (mgr.14195) 533 : audit 0 from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:24.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:23 smithi153 bash[14340]: audit 2024-03-12T02:25:22.731847+0000 mgr.smithi153.asomxr (mgr.14195) 533 : audit 0 from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:24.537 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:25:25.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:24 smithi163 bash[22347]: cluster 2024-03-12T02:25:23.709436+0000 mgr.smithi153.asomxr (mgr.14195) 534 : cluster 0 pgmap v444: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:25.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:24 smithi153 bash[14340]: cluster 2024-03-12T02:25:23.709436+0000 mgr.smithi153.asomxr (mgr.14195) 534 : cluster 0 pgmap v444: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:27.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:26 smithi163 bash[22347]: cluster 2024-03-12T02:25:25.709896+0000 mgr.smithi153.asomxr (mgr.14195) 535 : cluster 0 pgmap v445: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:27.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:26 smithi153 bash[14340]: cluster 2024-03-12T02:25:25.709896+0000 mgr.smithi153.asomxr (mgr.14195) 535 : cluster 0 pgmap v445: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:29.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:28 smithi163 bash[22347]: cluster 2024-03-12T02:25:27.710503+0000 mgr.smithi153.asomxr (mgr.14195) 536 : cluster 0 pgmap v446: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:29.293 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:25:29.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:28 smithi153 bash[14340]: cluster 2024-03-12T02:25:27.710503+0000 mgr.smithi153.asomxr (mgr.14195) 536 : cluster 0 pgmap v446: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:31.168 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:30 smithi153 bash[14340]: cluster 2024-03-12T02:25:29.711197+0000 mgr.smithi153.asomxr (mgr.14195) 537 : cluster 0 pgmap v447: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:31.168 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:30 smithi153 bash[14340]: audit 2024-03-12T02:25:30.397359+0000 mon.smithi153 (mon.0) 887 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:25:31.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:30 smithi163 bash[22347]: cluster 2024-03-12T02:25:29.711197+0000 mgr.smithi153.asomxr (mgr.14195) 537 : cluster 0 pgmap v447: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:31.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:30 smithi163 bash[22347]: audit 2024-03-12T02:25:30.397359+0000 mon.smithi153 (mon.0) 887 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:25:32.522 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:25:32.523 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:25:33.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:32 smithi153 bash[14340]: cluster 2024-03-12T02:25:31.711858+0000 mgr.smithi153.asomxr (mgr.14195) 538 : cluster 0 pgmap v448: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:33.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:32 smithi163 bash[22347]: cluster 2024-03-12T02:25:31.711858+0000 mgr.smithi153.asomxr (mgr.14195) 538 : cluster 0 pgmap v448: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:33.291 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:25:34.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:33 smithi163 bash[22347]: audit 2024-03-12T02:25:32.505806+0000 mgr.smithi153.asomxr (mgr.14195) 539 : audit 0 from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:34.292 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:25:34.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:33 smithi153 bash[14340]: audit 2024-03-12T02:25:32.505806+0000 mgr.smithi153.asomxr (mgr.14195) 539 : audit 0 from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:35.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:34 smithi163 bash[22347]: cluster 2024-03-12T02:25:33.712485+0000 mgr.smithi153.asomxr (mgr.14195) 540 : cluster 0 pgmap v449: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:35.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:34 smithi153 bash[14340]: cluster 2024-03-12T02:25:33.712485+0000 mgr.smithi153.asomxr (mgr.14195) 540 : cluster 0 pgmap v449: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:37.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:36 smithi163 bash[22347]: cluster 2024-03-12T02:25:35.713154+0000 mgr.smithi153.asomxr (mgr.14195) 541 : cluster 0 pgmap v450: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:37.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:36 smithi153 bash[14340]: cluster 2024-03-12T02:25:35.713154+0000 mgr.smithi153.asomxr (mgr.14195) 541 : cluster 0 pgmap v450: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:39.046 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:25:39.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:38 smithi163 bash[22347]: cluster 2024-03-12T02:25:37.713746+0000 mgr.smithi153.asomxr (mgr.14195) 542 : cluster 0 pgmap v451: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:39.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:38 smithi153 bash[14340]: cluster 2024-03-12T02:25:37.713746+0000 mgr.smithi153.asomxr (mgr.14195) 542 : cluster 0 pgmap v451: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:41.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:40 smithi163 bash[22347]: cluster 2024-03-12T02:25:39.714373+0000 mgr.smithi153.asomxr (mgr.14195) 543 : cluster 0 pgmap v452: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:41.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:40 smithi153 bash[14340]: cluster 2024-03-12T02:25:39.714373+0000 mgr.smithi153.asomxr (mgr.14195) 543 : cluster 0 pgmap v452: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:42.204 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:25:42.204 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:25:42.964 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:25:43.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:42 smithi163 bash[22347]: cluster 2024-03-12T02:25:41.714791+0000 mgr.smithi153.asomxr (mgr.14195) 544 : cluster 0 pgmap v453: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:43.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:42 smithi153 bash[14340]: cluster 2024-03-12T02:25:41.714791+0000 mgr.smithi153.asomxr (mgr.14195) 544 : cluster 0 pgmap v453: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:43.965 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:25:43.979 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:43 smithi153 bash[14340]: audit 2024-03-12T02:25:42.186488+0000 mgr.smithi153.asomxr (mgr.14195) 545 : audit 0 from='client.14690 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:44.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:43 smithi163 bash[22347]: audit 2024-03-12T02:25:42.186488+0000 mgr.smithi153.asomxr (mgr.14195) 545 : audit 0 from='client.14690 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:45.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:44 smithi163 bash[22347]: cluster 2024-03-12T02:25:43.715269+0000 mgr.smithi153.asomxr (mgr.14195) 546 : cluster 0 pgmap v454: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:45.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:44 smithi153 bash[14340]: cluster 2024-03-12T02:25:43.715269+0000 mgr.smithi153.asomxr (mgr.14195) 546 : cluster 0 pgmap v454: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:46.216 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:45 smithi163 bash[22347]: audit 2024-03-12T02:25:45.397802+0000 mon.smithi153 (mon.0) 888 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:25:46.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:45 smithi153 bash[14340]: audit 2024-03-12T02:25:45.397802+0000 mon.smithi153 (mon.0) 888 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:25:47.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:46 smithi163 bash[22347]: cluster 2024-03-12T02:25:45.715890+0000 mgr.smithi153.asomxr (mgr.14195) 547 : cluster 0 pgmap v455: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:47.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:46 smithi153 bash[14340]: cluster 2024-03-12T02:25:45.715890+0000 mgr.smithi153.asomxr (mgr.14195) 547 : cluster 0 pgmap v455: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:48.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:47 smithi163 bash[22347]: audit 2024-03-12T02:25:47.721950+0000 mon.smithi153 (mon.0) 889 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:25:48.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:47 smithi153 bash[14340]: audit 2024-03-12T02:25:47.721950+0000 mon.smithi153 (mon.0) 889 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:25:48.717 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:25:49.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: cluster 2024-03-12T02:25:47.716512+0000 mgr.smithi153.asomxr (mgr.14195) 548 : cluster 0 pgmap v456: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:49.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.302958+0000 mon.smithi153 (mon.0) 890 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.303527+0000 mon.smithi153 (mon.0) 891 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.305292+0000 mon.smithi153 (mon.0) 892 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.305725+0000 mon.smithi153 (mon.0) 893 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.307266+0000 mon.smithi153 (mon.0) 894 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.307664+0000 mon.smithi153 (mon.0) 895 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.308737+0000 mon.smithi153 (mon.0) 896 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.309097+0000 mon.smithi153 (mon.0) 897 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.316369+0000 mon.smithi153 (mon.0) 898 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.321981+0000 mon.smithi153 (mon.0) 899 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.323635+0000 mon.smithi153 (mon.0) 900 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.324763+0000 mon.smithi153 (mon.0) 901 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.331433+0000 mon.smithi153 (mon.0) 902 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:25:49.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:48 smithi153 bash[14340]: audit 2024-03-12T02:25:48.337707+0000 mon.smithi153 (mon.0) 903 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: cluster 2024-03-12T02:25:47.716512+0000 mgr.smithi153.asomxr (mgr.14195) 548 : cluster 0 pgmap v456: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.302958+0000 mon.smithi153 (mon.0) 890 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.303527+0000 mon.smithi153 (mon.0) 891 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.305292+0000 mon.smithi153 (mon.0) 892 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.305725+0000 mon.smithi153 (mon.0) 893 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.307266+0000 mon.smithi153 (mon.0) 894 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.307664+0000 mon.smithi153 (mon.0) 895 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.308737+0000 mon.smithi153 (mon.0) 896 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.309097+0000 mon.smithi153 (mon.0) 897 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.316369+0000 mon.smithi153 (mon.0) 898 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.321981+0000 mon.smithi153 (mon.0) 899 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:25:49.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.323635+0000 mon.smithi153 (mon.0) 900 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:25:49.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.324763+0000 mon.smithi153 (mon.0) 901 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:25:49.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.331433+0000 mon.smithi153 (mon.0) 902 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:25:49.235 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:48 smithi163 bash[22347]: audit 2024-03-12T02:25:48.337707+0000 mon.smithi153 (mon.0) 903 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:25:50.203 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:49 smithi153 bash[14340]: cephadm 2024-03-12T02:25:48.309868+0000 mgr.smithi153.asomxr (mgr.14195) 549 : cephadm 1 Adjusting osd_memory_target on smithi153 to 2492M 2024-03-12T02:25:50.203 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:49 smithi153 bash[14340]: cephadm 2024-03-12T02:25:48.310214+0000 mgr.smithi153.asomxr (mgr.14195) 550 : cephadm 1 Adjusting osd_memory_target on smithi163 to 2930M 2024-03-12T02:25:50.203 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:49 smithi153 bash[14340]: cluster 2024-03-12T02:25:48.326796+0000 mgr.smithi153.asomxr (mgr.14195) 551 : cluster 0 pgmap v457: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:50.203 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:49 smithi153 bash[14340]: cluster 2024-03-12T02:25:48.327039+0000 mgr.smithi153.asomxr (mgr.14195) 552 : cluster 0 pgmap v458: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:50.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:49 smithi163 bash[22347]: cephadm 2024-03-12T02:25:48.309868+0000 mgr.smithi153.asomxr (mgr.14195) 549 : cephadm 1 Adjusting osd_memory_target on smithi153 to 2492M 2024-03-12T02:25:50.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:49 smithi163 bash[22347]: cephadm 2024-03-12T02:25:48.310214+0000 mgr.smithi153.asomxr (mgr.14195) 550 : cephadm 1 Adjusting osd_memory_target on smithi163 to 2930M 2024-03-12T02:25:50.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:49 smithi163 bash[22347]: cluster 2024-03-12T02:25:48.326796+0000 mgr.smithi153.asomxr (mgr.14195) 551 : cluster 0 pgmap v457: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:50.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:49 smithi163 bash[22347]: cluster 2024-03-12T02:25:48.327039+0000 mgr.smithi153.asomxr (mgr.14195) 552 : cluster 0 pgmap v458: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:52.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:51 smithi153 bash[14340]: cluster 2024-03-12T02:25:50.327461+0000 mgr.smithi153.asomxr (mgr.14195) 553 : cluster 0 pgmap v459: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:52.177 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:25:52.178 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:25:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:51 smithi163 bash[22347]: cluster 2024-03-12T02:25:50.327461+0000 mgr.smithi153.asomxr (mgr.14195) 553 : cluster 0 pgmap v459: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:52.986 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:25:53.987 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:25:54.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:53 smithi163 bash[22347]: audit 2024-03-12T02:25:52.161797+0000 mgr.smithi153.asomxr (mgr.14195) 554 : audit 0 from='client.14694 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:54.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:53 smithi163 bash[22347]: cluster 2024-03-12T02:25:52.328126+0000 mgr.smithi153.asomxr (mgr.14195) 555 : cluster 0 pgmap v460: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:54.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:53 smithi153 bash[14340]: audit 2024-03-12T02:25:52.161797+0000 mgr.smithi153.asomxr (mgr.14195) 554 : audit 0 from='client.14694 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:25:54.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:53 smithi153 bash[14340]: cluster 2024-03-12T02:25:52.328126+0000 mgr.smithi153.asomxr (mgr.14195) 555 : cluster 0 pgmap v460: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:56.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:55 smithi163 bash[22347]: cluster 2024-03-12T02:25:54.328900+0000 mgr.smithi153.asomxr (mgr.14195) 556 : cluster 0 pgmap v461: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:56.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:55 smithi153 bash[14340]: cluster 2024-03-12T02:25:54.328900+0000 mgr.smithi153.asomxr (mgr.14195) 556 : cluster 0 pgmap v461: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:58.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:57 smithi163 bash[22347]: cluster 2024-03-12T02:25:56.329671+0000 mgr.smithi153.asomxr (mgr.14195) 557 : cluster 0 pgmap v462: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:58.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:57 smithi153 bash[14340]: cluster 2024-03-12T02:25:56.329671+0000 mgr.smithi153.asomxr (mgr.14195) 557 : cluster 0 pgmap v462: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:25:58.735 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:26:00.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:25:59 smithi153 bash[14340]: cluster 2024-03-12T02:25:58.330298+0000 mgr.smithi153.asomxr (mgr.14195) 558 : cluster 0 pgmap v463: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:00.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:25:59 smithi163 bash[22347]: cluster 2024-03-12T02:25:58.330298+0000 mgr.smithi153.asomxr (mgr.14195) 558 : cluster 0 pgmap v463: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:00 smithi163 bash[22347]: audit 2024-03-12T02:26:00.398214+0000 mon.smithi153 (mon.0) 904 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:26:01.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:00 smithi153 bash[14340]: audit 2024-03-12T02:26:00.398214+0000 mon.smithi153 (mon.0) 904 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:26:01.979 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:26:01.979 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:26:02.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:01 smithi163 bash[22347]: cluster 2024-03-12T02:26:00.330999+0000 mgr.smithi153.asomxr (mgr.14195) 559 : cluster 0 pgmap v464: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:02.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:01 smithi153 bash[14340]: cluster 2024-03-12T02:26:00.330999+0000 mgr.smithi153.asomxr (mgr.14195) 559 : cluster 0 pgmap v464: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:02.788 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:26:03.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:02 smithi153 bash[14340]: audit 2024-03-12T02:26:01.963139+0000 mgr.smithi153.asomxr (mgr.14195) 560 : audit 0 from='client.14698 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:03.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:02 smithi163 bash[22347]: audit 2024-03-12T02:26:01.963139+0000 mgr.smithi153.asomxr (mgr.14195) 560 : audit 0 from='client.14698 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:03.790 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:26:04.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:03 smithi153 bash[14340]: cluster 2024-03-12T02:26:02.331735+0000 mgr.smithi153.asomxr (mgr.14195) 561 : cluster 0 pgmap v465: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:04.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:03 smithi163 bash[22347]: cluster 2024-03-12T02:26:02.331735+0000 mgr.smithi153.asomxr (mgr.14195) 561 : cluster 0 pgmap v465: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:06.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:05 smithi163 bash[22347]: cluster 2024-03-12T02:26:04.332444+0000 mgr.smithi153.asomxr (mgr.14195) 562 : cluster 0 pgmap v466: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:06.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:05 smithi153 bash[14340]: cluster 2024-03-12T02:26:04.332444+0000 mgr.smithi153.asomxr (mgr.14195) 562 : cluster 0 pgmap v466: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:08.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:07 smithi153 bash[14340]: cluster 2024-03-12T02:26:06.333103+0000 mgr.smithi153.asomxr (mgr.14195) 563 : cluster 0 pgmap v467: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:08.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:07 smithi163 bash[22347]: cluster 2024-03-12T02:26:06.333103+0000 mgr.smithi153.asomxr (mgr.14195) 563 : cluster 0 pgmap v467: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:08.543 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:26:10.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:09 smithi153 bash[14340]: cluster 2024-03-12T02:26:08.333700+0000 mgr.smithi153.asomxr (mgr.14195) 564 : cluster 0 pgmap v468: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:10.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:09 smithi163 bash[22347]: cluster 2024-03-12T02:26:08.333700+0000 mgr.smithi153.asomxr (mgr.14195) 564 : cluster 0 pgmap v468: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:11.866 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:26:11.866 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:26:12.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:11 smithi153 bash[14340]: cluster 2024-03-12T02:26:10.334359+0000 mgr.smithi153.asomxr (mgr.14195) 565 : cluster 0 pgmap v469: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:11 smithi163 bash[22347]: cluster 2024-03-12T02:26:10.334359+0000 mgr.smithi153.asomxr (mgr.14195) 565 : cluster 0 pgmap v469: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:12.675 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:26:13.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:12 smithi153 bash[14340]: audit 2024-03-12T02:26:11.849402+0000 mgr.smithi153.asomxr (mgr.14195) 566 : audit 0 from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:13.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:12 smithi163 bash[22347]: audit 2024-03-12T02:26:11.849402+0000 mgr.smithi153.asomxr (mgr.14195) 566 : audit 0 from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:13.676 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:26:14.273 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:13 smithi153 bash[14340]: cluster 2024-03-12T02:26:12.334776+0000 mgr.smithi153.asomxr (mgr.14195) 567 : cluster 0 pgmap v470: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:14 smithi163 bash[22347]: cluster 2024-03-12T02:26:12.334776+0000 mgr.smithi153.asomxr (mgr.14195) 567 : cluster 0 pgmap v470: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:16.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:16 smithi153 bash[14340]: cluster 2024-03-12T02:26:14.335356+0000 mgr.smithi153.asomxr (mgr.14195) 568 : cluster 0 pgmap v471: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:16.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:16 smithi153 bash[14340]: audit 2024-03-12T02:26:15.398583+0000 mon.smithi153 (mon.0) 905 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:26:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:16 smithi163 bash[22347]: cluster 2024-03-12T02:26:14.335356+0000 mgr.smithi153.asomxr (mgr.14195) 568 : cluster 0 pgmap v471: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:16 smithi163 bash[22347]: audit 2024-03-12T02:26:15.398583+0000 mon.smithi153 (mon.0) 905 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:26:18.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:18 smithi153 bash[14340]: cluster 2024-03-12T02:26:16.336013+0000 mgr.smithi153.asomxr (mgr.14195) 569 : cluster 0 pgmap v472: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:18.427 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:26:18.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:18 smithi163 bash[22347]: cluster 2024-03-12T02:26:16.336013+0000 mgr.smithi153.asomxr (mgr.14195) 569 : cluster 0 pgmap v472: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:20.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:20 smithi153 bash[14340]: cluster 2024-03-12T02:26:18.336615+0000 mgr.smithi153.asomxr (mgr.14195) 570 : cluster 0 pgmap v473: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:20 smithi163 bash[22347]: cluster 2024-03-12T02:26:18.336615+0000 mgr.smithi153.asomxr (mgr.14195) 570 : cluster 0 pgmap v473: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:21.632 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:26:21.633 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:26:22.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:22 smithi163 bash[22347]: cluster 2024-03-12T02:26:20.337442+0000 mgr.smithi153.asomxr (mgr.14195) 571 : cluster 0 pgmap v474: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:22.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:22 smithi153 bash[14340]: cluster 2024-03-12T02:26:20.337442+0000 mgr.smithi153.asomxr (mgr.14195) 571 : cluster 0 pgmap v474: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:22.426 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:26:23.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:23 smithi153 bash[14340]: audit 2024-03-12T02:26:21.618751+0000 mgr.smithi153.asomxr (mgr.14195) 572 : audit 0 from='client.14706 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:23.427 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:26:23.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:23 smithi163 bash[22347]: audit 2024-03-12T02:26:21.618751+0000 mgr.smithi153.asomxr (mgr.14195) 572 : audit 0 from='client.14706 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:24.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:24 smithi153 bash[14340]: cluster 2024-03-12T02:26:22.338058+0000 mgr.smithi153.asomxr (mgr.14195) 573 : cluster 0 pgmap v475: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:24 smithi163 bash[22347]: cluster 2024-03-12T02:26:22.338058+0000 mgr.smithi153.asomxr (mgr.14195) 573 : cluster 0 pgmap v475: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:25.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:25 smithi163 bash[22347]: cluster 2024-03-12T02:26:24.338705+0000 mgr.smithi153.asomxr (mgr.14195) 574 : cluster 0 pgmap v476: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:25.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:25 smithi153 bash[14340]: cluster 2024-03-12T02:26:24.338705+0000 mgr.smithi153.asomxr (mgr.14195) 574 : cluster 0 pgmap v476: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:27.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:27 smithi163 bash[22347]: cluster 2024-03-12T02:26:26.339365+0000 mgr.smithi153.asomxr (mgr.14195) 575 : cluster 0 pgmap v477: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:27.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:27 smithi153 bash[14340]: cluster 2024-03-12T02:26:26.339365+0000 mgr.smithi153.asomxr (mgr.14195) 575 : cluster 0 pgmap v477: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:28.176 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:26:30.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:29 smithi153 bash[14340]: cluster 2024-03-12T02:26:28.339904+0000 mgr.smithi153.asomxr (mgr.14195) 576 : cluster 0 pgmap v478: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:30.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:29 smithi163 bash[22347]: cluster 2024-03-12T02:26:28.339904+0000 mgr.smithi153.asomxr (mgr.14195) 576 : cluster 0 pgmap v478: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:31.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:30 smithi153 bash[14340]: audit 2024-03-12T02:26:30.398405+0000 mon.smithi153 (mon.0) 906 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:26:31.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:30 smithi163 bash[22347]: audit 2024-03-12T02:26:30.398405+0000 mon.smithi153 (mon.0) 906 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:26:31.443 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:26:31.443 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:26:32.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:31 smithi153 bash[14340]: cluster 2024-03-12T02:26:30.340500+0000 mgr.smithi153.asomxr (mgr.14195) 577 : cluster 0 pgmap v479: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:32.145 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:31 smithi163 bash[22347]: cluster 2024-03-12T02:26:30.340500+0000 mgr.smithi153.asomxr (mgr.14195) 577 : cluster 0 pgmap v479: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:32.286 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:26:33.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:32 smithi153 bash[14340]: audit 2024-03-12T02:26:31.431381+0000 mgr.smithi153.asomxr (mgr.14195) 578 : audit 0 from='client.14710 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:33.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:32 smithi163 bash[22347]: audit 2024-03-12T02:26:31.431381+0000 mgr.smithi153.asomxr (mgr.14195) 578 : audit 0 from='client.14710 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:33.287 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:26:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:33 smithi153 bash[14340]: cluster 2024-03-12T02:26:32.341204+0000 mgr.smithi153.asomxr (mgr.14195) 579 : cluster 0 pgmap v480: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:34.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:33 smithi163 bash[22347]: cluster 2024-03-12T02:26:32.341204+0000 mgr.smithi153.asomxr (mgr.14195) 579 : cluster 0 pgmap v480: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:36.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:35 smithi163 bash[22347]: cluster 2024-03-12T02:26:34.341928+0000 mgr.smithi153.asomxr (mgr.14195) 580 : cluster 0 pgmap v481: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:36.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:35 smithi153 bash[14340]: cluster 2024-03-12T02:26:34.341928+0000 mgr.smithi153.asomxr (mgr.14195) 580 : cluster 0 pgmap v481: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:38.038 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:26:38.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:37 smithi163 bash[22347]: cluster 2024-03-12T02:26:36.342717+0000 mgr.smithi153.asomxr (mgr.14195) 581 : cluster 0 pgmap v482: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:38.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:37 smithi153 bash[14340]: cluster 2024-03-12T02:26:36.342717+0000 mgr.smithi153.asomxr (mgr.14195) 581 : cluster 0 pgmap v482: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:40.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:39 smithi153 bash[14340]: cluster 2024-03-12T02:26:38.343176+0000 mgr.smithi153.asomxr (mgr.14195) 582 : cluster 0 pgmap v483: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:40.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:39 smithi163 bash[22347]: cluster 2024-03-12T02:26:38.343176+0000 mgr.smithi153.asomxr (mgr.14195) 582 : cluster 0 pgmap v483: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:41.422 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:26:41.422 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:26:42.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:41 smithi153 bash[14340]: cluster 2024-03-12T02:26:40.343539+0000 mgr.smithi153.asomxr (mgr.14195) 583 : cluster 0 pgmap v484: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:42.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:41 smithi163 bash[22347]: cluster 2024-03-12T02:26:40.343539+0000 mgr.smithi153.asomxr (mgr.14195) 583 : cluster 0 pgmap v484: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:42.264 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:26:43.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:42 smithi163 bash[22347]: audit 2024-03-12T02:26:41.406373+0000 mgr.smithi153.asomxr (mgr.14195) 584 : audit 0 from='client.14714 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:43.265 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:26:43.276 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:42 smithi153 bash[14340]: audit 2024-03-12T02:26:41.406373+0000 mgr.smithi153.asomxr (mgr.14195) 584 : audit 0 from='client.14714 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:44.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:43 smithi163 bash[22347]: cluster 2024-03-12T02:26:42.344120+0000 mgr.smithi153.asomxr (mgr.14195) 585 : cluster 0 pgmap v485: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:44.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:43 smithi153 bash[14340]: cluster 2024-03-12T02:26:42.344120+0000 mgr.smithi153.asomxr (mgr.14195) 585 : cluster 0 pgmap v485: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:46.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:45 smithi163 bash[22347]: cluster 2024-03-12T02:26:44.344734+0000 mgr.smithi153.asomxr (mgr.14195) 586 : cluster 0 pgmap v486: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:46.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:45 smithi163 bash[22347]: audit 2024-03-12T02:26:45.399302+0000 mon.smithi153 (mon.0) 907 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:26:46.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:45 smithi153 bash[14340]: cluster 2024-03-12T02:26:44.344734+0000 mgr.smithi153.asomxr (mgr.14195) 586 : cluster 0 pgmap v486: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:46.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:45 smithi153 bash[14340]: audit 2024-03-12T02:26:45.399302+0000 mon.smithi153 (mon.0) 907 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:26:48.011 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:26:48.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:47 smithi163 bash[22347]: cluster 2024-03-12T02:26:46.345455+0000 mgr.smithi153.asomxr (mgr.14195) 587 : cluster 0 pgmap v487: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:48.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:47 smithi153 bash[14340]: cluster 2024-03-12T02:26:46.345455+0000 mgr.smithi153.asomxr (mgr.14195) 587 : cluster 0 pgmap v487: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:49.121 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:48 smithi153 bash[14340]: audit 2024-03-12T02:26:48.432560+0000 mon.smithi153 (mon.0) 908 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:26:49.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:48 smithi163 bash[22347]: audit 2024-03-12T02:26:48.432560+0000 mon.smithi153 (mon.0) 908 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:26:50.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:49 smithi163 bash[22347]: cluster 2024-03-12T02:26:48.346259+0000 mgr.smithi153.asomxr (mgr.14195) 588 : cluster 0 pgmap v488: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:50.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:49 smithi163 bash[22347]: audit 2024-03-12T02:26:48.958903+0000 mon.smithi153 (mon.0) 909 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:26:50.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:49 smithi163 bash[22347]: audit 2024-03-12T02:26:48.959950+0000 mon.smithi153 (mon.0) 910 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:26:50.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:49 smithi163 bash[22347]: cluster 2024-03-12T02:26:48.961942+0000 mgr.smithi153.asomxr (mgr.14195) 589 : cluster 0 pgmap v489: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:50.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:49 smithi163 bash[22347]: cluster 2024-03-12T02:26:48.962123+0000 mgr.smithi153.asomxr (mgr.14195) 590 : cluster 0 pgmap v490: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:50.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:49 smithi163 bash[22347]: audit 2024-03-12T02:26:48.969777+0000 mon.smithi153 (mon.0) 911 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:26:50.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:49 smithi163 bash[22347]: audit 2024-03-12T02:26:48.973982+0000 mon.smithi153 (mon.0) 912 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:26:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:49 smithi153 bash[14340]: cluster 2024-03-12T02:26:48.346259+0000 mgr.smithi153.asomxr (mgr.14195) 588 : cluster 0 pgmap v488: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:49 smithi153 bash[14340]: audit 2024-03-12T02:26:48.958903+0000 mon.smithi153 (mon.0) 909 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:26:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:49 smithi153 bash[14340]: audit 2024-03-12T02:26:48.959950+0000 mon.smithi153 (mon.0) 910 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:26:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:49 smithi153 bash[14340]: cluster 2024-03-12T02:26:48.961942+0000 mgr.smithi153.asomxr (mgr.14195) 589 : cluster 0 pgmap v489: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:49 smithi153 bash[14340]: cluster 2024-03-12T02:26:48.962123+0000 mgr.smithi153.asomxr (mgr.14195) 590 : cluster 0 pgmap v490: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:49 smithi153 bash[14340]: audit 2024-03-12T02:26:48.969777+0000 mon.smithi153 (mon.0) 911 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:26:50.326 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:49 smithi153 bash[14340]: audit 2024-03-12T02:26:48.973982+0000 mon.smithi153 (mon.0) 912 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:26:51.332 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:26:51.332 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:26:52.149 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:26:52.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:52 smithi153 bash[14340]: cluster 2024-03-12T02:26:50.962583+0000 mgr.smithi153.asomxr (mgr.14195) 591 : cluster 0 pgmap v491: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:52 smithi163 bash[22347]: cluster 2024-03-12T02:26:50.962583+0000 mgr.smithi153.asomxr (mgr.14195) 591 : cluster 0 pgmap v491: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:53.149 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:26:53.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:53 smithi153 bash[14340]: audit 2024-03-12T02:26:51.315675+0000 mgr.smithi153.asomxr (mgr.14195) 592 : audit 0 from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:53.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:53 smithi163 bash[22347]: audit 2024-03-12T02:26:51.315675+0000 mgr.smithi153.asomxr (mgr.14195) 592 : audit 0 from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:26:54.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:54 smithi153 bash[14340]: cluster 2024-03-12T02:26:52.963194+0000 mgr.smithi153.asomxr (mgr.14195) 593 : cluster 0 pgmap v492: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:54 smithi163 bash[22347]: cluster 2024-03-12T02:26:52.963194+0000 mgr.smithi153.asomxr (mgr.14195) 593 : cluster 0 pgmap v492: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:56.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:56 smithi153 bash[14340]: cluster 2024-03-12T02:26:54.963806+0000 mgr.smithi153.asomxr (mgr.14195) 594 : cluster 0 pgmap v493: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:56.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:56 smithi163 bash[22347]: cluster 2024-03-12T02:26:54.963806+0000 mgr.smithi153.asomxr (mgr.14195) 594 : cluster 0 pgmap v493: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:57.902 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:26:58.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:26:58 smithi153 bash[14340]: cluster 2024-03-12T02:26:56.964411+0000 mgr.smithi153.asomxr (mgr.14195) 595 : cluster 0 pgmap v494: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:26:58.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:26:58 smithi163 bash[22347]: cluster 2024-03-12T02:26:56.964411+0000 mgr.smithi153.asomxr (mgr.14195) 595 : cluster 0 pgmap v494: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:00.175 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:00 smithi153 bash[14340]: cluster 2024-03-12T02:26:58.965031+0000 mgr.smithi153.asomxr (mgr.14195) 596 : cluster 0 pgmap v495: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:00.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:00 smithi163 bash[22347]: cluster 2024-03-12T02:26:58.965031+0000 mgr.smithi153.asomxr (mgr.14195) 596 : cluster 0 pgmap v495: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:01.114 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:27:01.114 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:27:01.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:01 smithi153 bash[14340]: audit 2024-03-12T02:27:00.399805+0000 mon.smithi153 (mon.0) 913 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:27:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:01 smithi163 bash[22347]: audit 2024-03-12T02:27:00.399805+0000 mon.smithi153 (mon.0) 913 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:27:01.994 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:27:02.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:02 smithi153 bash[14340]: cluster 2024-03-12T02:27:00.965646+0000 mgr.smithi153.asomxr (mgr.14195) 597 : cluster 0 pgmap v496: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:02.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:02 smithi153 bash[14340]: audit 2024-03-12T02:27:01.098363+0000 mgr.smithi153.asomxr (mgr.14195) 598 : audit 0 from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:02 smithi163 bash[22347]: cluster 2024-03-12T02:27:00.965646+0000 mgr.smithi153.asomxr (mgr.14195) 597 : cluster 0 pgmap v496: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:02 smithi163 bash[22347]: audit 2024-03-12T02:27:01.098363+0000 mgr.smithi153.asomxr (mgr.14195) 598 : audit 0 from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:02.996 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:27:04.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:03 smithi153 bash[14340]: cluster 2024-03-12T02:27:02.966208+0000 mgr.smithi153.asomxr (mgr.14195) 599 : cluster 0 pgmap v497: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:04.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:03 smithi163 bash[22347]: cluster 2024-03-12T02:27:02.966208+0000 mgr.smithi153.asomxr (mgr.14195) 599 : cluster 0 pgmap v497: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:06.326 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:06 smithi153 bash[14340]: cluster 2024-03-12T02:27:04.966995+0000 mgr.smithi153.asomxr (mgr.14195) 600 : cluster 0 pgmap v498: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:06.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:06 smithi163 bash[22347]: cluster 2024-03-12T02:27:04.966995+0000 mgr.smithi153.asomxr (mgr.14195) 600 : cluster 0 pgmap v498: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:07.749 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:27:08.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:08 smithi153 bash[14340]: cluster 2024-03-12T02:27:06.967650+0000 mgr.smithi153.asomxr (mgr.14195) 601 : cluster 0 pgmap v499: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:08.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:08 smithi163 bash[22347]: cluster 2024-03-12T02:27:06.967650+0000 mgr.smithi153.asomxr (mgr.14195) 601 : cluster 0 pgmap v499: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:10.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:10 smithi153 bash[14340]: cluster 2024-03-12T02:27:08.968229+0000 mgr.smithi153.asomxr (mgr.14195) 602 : cluster 0 pgmap v500: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:10.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:10 smithi163 bash[22347]: cluster 2024-03-12T02:27:08.968229+0000 mgr.smithi153.asomxr (mgr.14195) 602 : cluster 0 pgmap v500: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:11.284 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:27:11.284 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:27:12.124 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:27:12.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:12 smithi153 bash[14340]: cluster 2024-03-12T02:27:10.968825+0000 mgr.smithi153.asomxr (mgr.14195) 603 : cluster 0 pgmap v501: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:12 smithi163 bash[22347]: cluster 2024-03-12T02:27:10.968825+0000 mgr.smithi153.asomxr (mgr.14195) 603 : cluster 0 pgmap v501: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:13.125 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:27:13.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:13 smithi163 bash[22347]: audit 2024-03-12T02:27:11.268854+0000 mgr.smithi153.asomxr (mgr.14195) 604 : audit 0 from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:13.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:13 smithi153 bash[14340]: audit 2024-03-12T02:27:11.268854+0000 mgr.smithi153.asomxr (mgr.14195) 604 : audit 0 from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:14.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:14 smithi153 bash[14340]: cluster 2024-03-12T02:27:12.969488+0000 mgr.smithi153.asomxr (mgr.14195) 605 : cluster 0 pgmap v502: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:14 smithi163 bash[22347]: cluster 2024-03-12T02:27:12.969488+0000 mgr.smithi153.asomxr (mgr.14195) 605 : cluster 0 pgmap v502: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:15.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:15 smithi153 bash[14340]: cluster 2024-03-12T02:27:14.970052+0000 mgr.smithi153.asomxr (mgr.14195) 606 : cluster 0 pgmap v503: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:15.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:15 smithi163 bash[22347]: cluster 2024-03-12T02:27:14.970052+0000 mgr.smithi153.asomxr (mgr.14195) 606 : cluster 0 pgmap v503: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:16.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:16 smithi153 bash[14340]: audit 2024-03-12T02:27:15.399686+0000 mon.smithi153 (mon.0) 914 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:27:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:16 smithi163 bash[22347]: audit 2024-03-12T02:27:15.399686+0000 mon.smithi153 (mon.0) 914 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:27:17.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:17 smithi153 bash[14340]: cluster 2024-03-12T02:27:16.970765+0000 mgr.smithi153.asomxr (mgr.14195) 607 : cluster 0 pgmap v504: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:17.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:17 smithi163 bash[22347]: cluster 2024-03-12T02:27:16.970765+0000 mgr.smithi153.asomxr (mgr.14195) 607 : cluster 0 pgmap v504: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:17.878 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:27:20.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:20 smithi153 bash[14340]: cluster 2024-03-12T02:27:18.971433+0000 mgr.smithi153.asomxr (mgr.14195) 608 : cluster 0 pgmap v505: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:20 smithi163 bash[22347]: cluster 2024-03-12T02:27:18.971433+0000 mgr.smithi153.asomxr (mgr.14195) 608 : cluster 0 pgmap v505: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:21.017 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:27:21.017 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:27:21.838 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:27:22.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:22 smithi153 bash[14340]: cluster 2024-03-12T02:27:20.972149+0000 mgr.smithi153.asomxr (mgr.14195) 609 : cluster 0 pgmap v506: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:22.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:22 smithi153 bash[14340]: audit 2024-03-12T02:27:21.000230+0000 mgr.smithi153.asomxr (mgr.14195) 610 : audit 0 from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:22.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:22 smithi163 bash[22347]: cluster 2024-03-12T02:27:20.972149+0000 mgr.smithi153.asomxr (mgr.14195) 609 : cluster 0 pgmap v506: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:22.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:22 smithi163 bash[22347]: audit 2024-03-12T02:27:21.000230+0000 mgr.smithi153.asomxr (mgr.14195) 610 : audit 0 from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:22.838 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:27:24.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:23 smithi163 bash[22347]: cluster 2024-03-12T02:27:22.972721+0000 mgr.smithi153.asomxr (mgr.14195) 611 : cluster 0 pgmap v507: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:24.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:23 smithi153 bash[14340]: cluster 2024-03-12T02:27:22.972721+0000 mgr.smithi153.asomxr (mgr.14195) 611 : cluster 0 pgmap v507: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:26.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:26 smithi153 bash[14340]: cluster 2024-03-12T02:27:24.973362+0000 mgr.smithi153.asomxr (mgr.14195) 612 : cluster 0 pgmap v508: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:26 smithi163 bash[22347]: cluster 2024-03-12T02:27:24.973362+0000 mgr.smithi153.asomxr (mgr.14195) 612 : cluster 0 pgmap v508: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:27.587 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:27:28.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:28 smithi153 bash[14340]: cluster 2024-03-12T02:27:26.974026+0000 mgr.smithi153.asomxr (mgr.14195) 613 : cluster 0 pgmap v509: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:28 smithi163 bash[22347]: cluster 2024-03-12T02:27:26.974026+0000 mgr.smithi153.asomxr (mgr.14195) 613 : cluster 0 pgmap v509: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:30.313 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:30 smithi153 bash[14340]: cluster 2024-03-12T02:27:28.974787+0000 mgr.smithi153.asomxr (mgr.14195) 614 : cluster 0 pgmap v510: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:30.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:30 smithi163 bash[22347]: cluster 2024-03-12T02:27:28.974787+0000 mgr.smithi153.asomxr (mgr.14195) 614 : cluster 0 pgmap v510: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:30.733 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:27:30.733 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:27:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:31 smithi153 bash[14340]: audit 2024-03-12T02:27:30.399800+0000 mon.smithi153 (mon.0) 915 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:27:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:31 smithi163 bash[22347]: audit 2024-03-12T02:27:30.399800+0000 mon.smithi153 (mon.0) 915 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:27:31.532 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:27:32.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:32 smithi153 bash[14340]: audit 2024-03-12T02:27:30.714885+0000 mgr.smithi153.asomxr (mgr.14195) 615 : audit 0 from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:32.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:32 smithi153 bash[14340]: cluster 2024-03-12T02:27:30.975421+0000 mgr.smithi153.asomxr (mgr.14195) 616 : cluster 0 pgmap v511: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:32 smithi163 bash[22347]: audit 2024-03-12T02:27:30.714885+0000 mgr.smithi153.asomxr (mgr.14195) 615 : audit 0 from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:32 smithi163 bash[22347]: cluster 2024-03-12T02:27:30.975421+0000 mgr.smithi153.asomxr (mgr.14195) 616 : cluster 0 pgmap v511: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:32.534 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:27:34.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:33 smithi163 bash[22347]: cluster 2024-03-12T02:27:32.976030+0000 mgr.smithi153.asomxr (mgr.14195) 617 : cluster 0 pgmap v512: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:34.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:33 smithi153 bash[14340]: cluster 2024-03-12T02:27:32.976030+0000 mgr.smithi153.asomxr (mgr.14195) 617 : cluster 0 pgmap v512: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:36.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:36 smithi153 bash[14340]: cluster 2024-03-12T02:27:34.976640+0000 mgr.smithi153.asomxr (mgr.14195) 618 : cluster 0 pgmap v513: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:36.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:36 smithi163 bash[22347]: cluster 2024-03-12T02:27:34.976640+0000 mgr.smithi153.asomxr (mgr.14195) 618 : cluster 0 pgmap v513: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:27:38.319 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:38 smithi153 bash[14340]: cluster 2024-03-12T02:27:36.977288+0000 mgr.smithi153.asomxr (mgr.14195) 619 : cluster 0 pgmap v514: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:38.443 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:38 smithi163 bash[22347]: cluster 2024-03-12T02:27:36.977288+0000 mgr.smithi153.asomxr (mgr.14195) 619 : cluster 0 pgmap v514: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:40.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:40 smithi153 bash[14340]: cluster 2024-03-12T02:27:38.977886+0000 mgr.smithi153.asomxr (mgr.14195) 620 : cluster 0 pgmap v515: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:40.396 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:27:40.397 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:27:40.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:40 smithi163 bash[22347]: cluster 2024-03-12T02:27:38.977886+0000 mgr.smithi153.asomxr (mgr.14195) 620 : cluster 0 pgmap v515: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:41.125 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:27:41.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:41 smithi153 bash[14340]: audit 2024-03-12T02:27:40.378041+0000 mgr.smithi153.asomxr (mgr.14195) 621 : audit 0 from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:41.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:41 smithi153 bash[14340]: cluster 2024-03-12T02:27:40.978486+0000 mgr.smithi153.asomxr (mgr.14195) 622 : cluster 0 pgmap v516: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:41.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:41 smithi163 bash[22347]: audit 2024-03-12T02:27:40.378041+0000 mgr.smithi153.asomxr (mgr.14195) 621 : audit 0 from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:41.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:41 smithi163 bash[22347]: cluster 2024-03-12T02:27:40.978486+0000 mgr.smithi153.asomxr (mgr.14195) 622 : cluster 0 pgmap v516: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:42.126 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:27:44.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:43 smithi163 bash[22347]: cluster 2024-03-12T02:27:42.979067+0000 mgr.smithi153.asomxr (mgr.14195) 623 : cluster 0 pgmap v517: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:44.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:43 smithi153 bash[14340]: cluster 2024-03-12T02:27:42.979067+0000 mgr.smithi153.asomxr (mgr.14195) 623 : cluster 0 pgmap v517: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:46.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:46 smithi153 bash[14340]: cluster 2024-03-12T02:27:44.979640+0000 mgr.smithi153.asomxr (mgr.14195) 624 : cluster 0 pgmap v518: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:46.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:46 smithi153 bash[14340]: audit 2024-03-12T02:27:45.400487+0000 mon.smithi153 (mon.0) 916 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:27:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:46 smithi163 bash[22347]: cluster 2024-03-12T02:27:44.979640+0000 mgr.smithi153.asomxr (mgr.14195) 624 : cluster 0 pgmap v518: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:46 smithi163 bash[22347]: audit 2024-03-12T02:27:45.400487+0000 mon.smithi153 (mon.0) 916 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:27:46.883 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:27:48.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:48 smithi153 bash[14340]: cluster 2024-03-12T02:27:46.980170+0000 mgr.smithi153.asomxr (mgr.14195) 625 : cluster 0 pgmap v519: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:48.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:48 smithi163 bash[22347]: cluster 2024-03-12T02:27:46.980170+0000 mgr.smithi153.asomxr (mgr.14195) 625 : cluster 0 pgmap v519: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:49.909 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:27:49.909 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:27:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:50 smithi153 bash[14340]: cluster 2024-03-12T02:27:48.980775+0000 mgr.smithi153.asomxr (mgr.14195) 626 : cluster 0 pgmap v520: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:50 smithi153 bash[14340]: audit 2024-03-12T02:27:49.064493+0000 mon.smithi153 (mon.0) 917 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:27:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:50 smithi153 bash[14340]: audit 2024-03-12T02:27:49.594750+0000 mon.smithi153 (mon.0) 918 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:27:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:50 smithi153 bash[14340]: audit 2024-03-12T02:27:49.596017+0000 mon.smithi153 (mon.0) 919 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:27:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:50 smithi153 bash[14340]: audit 2024-03-12T02:27:49.604808+0000 mon.smithi153 (mon.0) 920 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:27:50.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:50 smithi153 bash[14340]: audit 2024-03-12T02:27:49.627386+0000 mon.smithi153 (mon.0) 921 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:27:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:50 smithi163 bash[22347]: cluster 2024-03-12T02:27:48.980775+0000 mgr.smithi153.asomxr (mgr.14195) 626 : cluster 0 pgmap v520: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:50 smithi163 bash[22347]: audit 2024-03-12T02:27:49.064493+0000 mon.smithi153 (mon.0) 917 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:27:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:50 smithi163 bash[22347]: audit 2024-03-12T02:27:49.594750+0000 mon.smithi153 (mon.0) 918 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:27:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:50 smithi163 bash[22347]: audit 2024-03-12T02:27:49.596017+0000 mon.smithi153 (mon.0) 919 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:27:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:50 smithi163 bash[22347]: audit 2024-03-12T02:27:49.604808+0000 mon.smithi153 (mon.0) 920 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:27:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:50 smithi163 bash[22347]: audit 2024-03-12T02:27:49.627386+0000 mon.smithi153 (mon.0) 921 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:27:50.686 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:27:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:51 smithi153 bash[14340]: cluster 2024-03-12T02:27:49.597942+0000 mgr.smithi153.asomxr (mgr.14195) 627 : cluster 0 pgmap v521: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:51 smithi153 bash[14340]: cluster 2024-03-12T02:27:49.598288+0000 mgr.smithi153.asomxr (mgr.14195) 628 : cluster 0 pgmap v522: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:51 smithi153 bash[14340]: audit 2024-03-12T02:27:49.889624+0000 mgr.smithi153.asomxr (mgr.14195) 629 : audit 0 from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:51 smithi163 bash[22347]: cluster 2024-03-12T02:27:49.597942+0000 mgr.smithi153.asomxr (mgr.14195) 627 : cluster 0 pgmap v521: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:51 smithi163 bash[22347]: cluster 2024-03-12T02:27:49.598288+0000 mgr.smithi153.asomxr (mgr.14195) 628 : cluster 0 pgmap v522: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:51 smithi163 bash[22347]: audit 2024-03-12T02:27:49.889624+0000 mgr.smithi153.asomxr (mgr.14195) 629 : audit 0 from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:27:51.687 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:27:52.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:52 smithi153 bash[14340]: cluster 2024-03-12T02:27:51.598807+0000 mgr.smithi153.asomxr (mgr.14195) 630 : cluster 0 pgmap v523: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:52 smithi163 bash[22347]: cluster 2024-03-12T02:27:51.598807+0000 mgr.smithi153.asomxr (mgr.14195) 630 : cluster 0 pgmap v523: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:55.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:54 smithi163 bash[22347]: cluster 2024-03-12T02:27:53.599437+0000 mgr.smithi153.asomxr (mgr.14195) 631 : cluster 0 pgmap v524: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:55.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:54 smithi153 bash[14340]: cluster 2024-03-12T02:27:53.599437+0000 mgr.smithi153.asomxr (mgr.14195) 631 : cluster 0 pgmap v524: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:56.436 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:27:57.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:56 smithi163 bash[22347]: cluster 2024-03-12T02:27:55.600115+0000 mgr.smithi153.asomxr (mgr.14195) 632 : cluster 0 pgmap v525: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:57.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:56 smithi153 bash[14340]: cluster 2024-03-12T02:27:55.600115+0000 mgr.smithi153.asomxr (mgr.14195) 632 : cluster 0 pgmap v525: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:59.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:27:58 smithi163 bash[22347]: cluster 2024-03-12T02:27:57.600872+0000 mgr.smithi153.asomxr (mgr.14195) 633 : cluster 0 pgmap v526: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:27:59.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:27:58 smithi153 bash[14340]: cluster 2024-03-12T02:27:57.600872+0000 mgr.smithi153.asomxr (mgr.14195) 633 : cluster 0 pgmap v526: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:00.158 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:28:00.158 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:28:00.988 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:28:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:00 smithi163 bash[22347]: cluster 2024-03-12T02:27:59.601384+0000 mgr.smithi153.asomxr (mgr.14195) 634 : cluster 0 pgmap v527: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:00 smithi163 bash[22347]: audit 2024-03-12T02:28:00.400406+0000 mon.smithi153 (mon.0) 922 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:28:01.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:00 smithi153 bash[14340]: cluster 2024-03-12T02:27:59.601384+0000 mgr.smithi153.asomxr (mgr.14195) 634 : cluster 0 pgmap v527: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:01.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:00 smithi153 bash[14340]: audit 2024-03-12T02:28:00.400406+0000 mon.smithi153 (mon.0) 922 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:28:01.989 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:28:02.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:01 smithi163 bash[22347]: audit 2024-03-12T02:28:00.141634+0000 mgr.smithi153.asomxr (mgr.14195) 635 : audit 0 from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:02.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:01 smithi153 bash[14340]: audit 2024-03-12T02:28:00.141634+0000 mgr.smithi153.asomxr (mgr.14195) 635 : audit 0 from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:03.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:02 smithi163 bash[22347]: cluster 2024-03-12T02:28:01.601966+0000 mgr.smithi153.asomxr (mgr.14195) 636 : cluster 0 pgmap v528: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:03.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:02 smithi153 bash[14340]: cluster 2024-03-12T02:28:01.601966+0000 mgr.smithi153.asomxr (mgr.14195) 636 : cluster 0 pgmap v528: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:05.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:04 smithi163 bash[22347]: cluster 2024-03-12T02:28:03.602599+0000 mgr.smithi153.asomxr (mgr.14195) 637 : cluster 0 pgmap v529: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:05.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:04 smithi153 bash[14340]: cluster 2024-03-12T02:28:03.602599+0000 mgr.smithi153.asomxr (mgr.14195) 637 : cluster 0 pgmap v529: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:06.736 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:28:07.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:06 smithi153 bash[14340]: cluster 2024-03-12T02:28:05.603188+0000 mgr.smithi153.asomxr (mgr.14195) 638 : cluster 0 pgmap v530: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:07.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:06 smithi163 bash[22347]: cluster 2024-03-12T02:28:05.603188+0000 mgr.smithi153.asomxr (mgr.14195) 638 : cluster 0 pgmap v530: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:09.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:08 smithi153 bash[14340]: cluster 2024-03-12T02:28:07.603819+0000 mgr.smithi153.asomxr (mgr.14195) 639 : cluster 0 pgmap v531: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:09.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:08 smithi163 bash[22347]: cluster 2024-03-12T02:28:07.603819+0000 mgr.smithi153.asomxr (mgr.14195) 639 : cluster 0 pgmap v531: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:10.082 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:28:10.082 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:28:10.901 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:28:11.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:10 smithi163 bash[22347]: cluster 2024-03-12T02:28:09.604490+0000 mgr.smithi153.asomxr (mgr.14195) 640 : cluster 0 pgmap v532: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:10 smithi153 bash[14340]: cluster 2024-03-12T02:28:09.604490+0000 mgr.smithi153.asomxr (mgr.14195) 640 : cluster 0 pgmap v532: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:11.902 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:28:12.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:11 smithi163 bash[22347]: audit 2024-03-12T02:28:10.070570+0000 mgr.smithi153.asomxr (mgr.14195) 641 : audit 0 from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:12.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:11 smithi153 bash[14340]: audit 2024-03-12T02:28:10.070570+0000 mgr.smithi153.asomxr (mgr.14195) 641 : audit 0 from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:13.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:12 smithi163 bash[22347]: cluster 2024-03-12T02:28:11.605095+0000 mgr.smithi153.asomxr (mgr.14195) 642 : cluster 0 pgmap v533: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:13.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:12 smithi153 bash[14340]: cluster 2024-03-12T02:28:11.605095+0000 mgr.smithi153.asomxr (mgr.14195) 642 : cluster 0 pgmap v533: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:15.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:14 smithi163 bash[22347]: cluster 2024-03-12T02:28:13.605698+0000 mgr.smithi153.asomxr (mgr.14195) 643 : cluster 0 pgmap v534: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:15.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:14 smithi153 bash[14340]: cluster 2024-03-12T02:28:13.605698+0000 mgr.smithi153.asomxr (mgr.14195) 643 : cluster 0 pgmap v534: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:16.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:15 smithi163 bash[22347]: audit 2024-03-12T02:28:15.401498+0000 mon.smithi153 (mon.0) 923 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:28:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:15 smithi153 bash[14340]: audit 2024-03-12T02:28:15.401498+0000 mon.smithi153 (mon.0) 923 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:28:16.662 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:28:17.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:16 smithi153 bash[14340]: cluster 2024-03-12T02:28:15.606325+0000 mgr.smithi153.asomxr (mgr.14195) 644 : cluster 0 pgmap v535: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:17.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:16 smithi163 bash[22347]: cluster 2024-03-12T02:28:15.606325+0000 mgr.smithi153.asomxr (mgr.14195) 644 : cluster 0 pgmap v535: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:19.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:18 smithi153 bash[14340]: cluster 2024-03-12T02:28:17.606977+0000 mgr.smithi153.asomxr (mgr.14195) 645 : cluster 0 pgmap v536: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:19.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:18 smithi163 bash[22347]: cluster 2024-03-12T02:28:17.606977+0000 mgr.smithi153.asomxr (mgr.14195) 645 : cluster 0 pgmap v536: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:20.081 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:28:20.081 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:28:20.889 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:28:21.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:20 smithi163 bash[22347]: cluster 2024-03-12T02:28:19.607517+0000 mgr.smithi153.asomxr (mgr.14195) 646 : cluster 0 pgmap v537: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:21.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:20 smithi153 bash[14340]: cluster 2024-03-12T02:28:19.607517+0000 mgr.smithi153.asomxr (mgr.14195) 646 : cluster 0 pgmap v537: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:21.890 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:28:22.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:21 smithi153 bash[14340]: audit 2024-03-12T02:28:20.063575+0000 mgr.smithi153.asomxr (mgr.14195) 647 : audit 0 from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:22.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:21 smithi163 bash[22347]: audit 2024-03-12T02:28:20.063575+0000 mgr.smithi153.asomxr (mgr.14195) 647 : audit 0 from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:23.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:22 smithi163 bash[22347]: cluster 2024-03-12T02:28:21.608101+0000 mgr.smithi153.asomxr (mgr.14195) 648 : cluster 0 pgmap v538: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:23.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:22 smithi153 bash[14340]: cluster 2024-03-12T02:28:21.608101+0000 mgr.smithi153.asomxr (mgr.14195) 648 : cluster 0 pgmap v538: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:25.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:24 smithi163 bash[22347]: cluster 2024-03-12T02:28:23.608735+0000 mgr.smithi153.asomxr (mgr.14195) 649 : cluster 0 pgmap v539: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:25.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:24 smithi153 bash[14340]: cluster 2024-03-12T02:28:23.608735+0000 mgr.smithi153.asomxr (mgr.14195) 649 : cluster 0 pgmap v539: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:26.644 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:28:27.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:26 smithi153 bash[14340]: cluster 2024-03-12T02:28:25.609423+0000 mgr.smithi153.asomxr (mgr.14195) 650 : cluster 0 pgmap v540: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:27.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:26 smithi163 bash[22347]: cluster 2024-03-12T02:28:25.609423+0000 mgr.smithi153.asomxr (mgr.14195) 650 : cluster 0 pgmap v540: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:29.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:28 smithi163 bash[22347]: cluster 2024-03-12T02:28:27.609938+0000 mgr.smithi153.asomxr (mgr.14195) 651 : cluster 0 pgmap v541: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:29.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:28 smithi153 bash[14340]: cluster 2024-03-12T02:28:27.609938+0000 mgr.smithi153.asomxr (mgr.14195) 651 : cluster 0 pgmap v541: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:29.944 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:28:29.944 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:28:30.755 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:28:31.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:30 smithi153 bash[14340]: cluster 2024-03-12T02:28:29.610518+0000 mgr.smithi153.asomxr (mgr.14195) 652 : cluster 0 pgmap v542: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:31.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:30 smithi153 bash[14340]: audit 2024-03-12T02:28:29.927174+0000 mgr.smithi153.asomxr (mgr.14195) 653 : audit 0 from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:31.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:30 smithi153 bash[14340]: audit 2024-03-12T02:28:30.401375+0000 mon.smithi153 (mon.0) 924 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:28:31.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:30 smithi163 bash[22347]: cluster 2024-03-12T02:28:29.610518+0000 mgr.smithi153.asomxr (mgr.14195) 652 : cluster 0 pgmap v542: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:31.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:30 smithi163 bash[22347]: audit 2024-03-12T02:28:29.927174+0000 mgr.smithi153.asomxr (mgr.14195) 653 : audit 0 from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:31.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:30 smithi163 bash[22347]: audit 2024-03-12T02:28:30.401375+0000 mon.smithi153 (mon.0) 924 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:28:31.756 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:28:33.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:32 smithi163 bash[22347]: cluster 2024-03-12T02:28:31.611118+0000 mgr.smithi153.asomxr (mgr.14195) 654 : cluster 0 pgmap v543: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:33.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:32 smithi153 bash[14340]: cluster 2024-03-12T02:28:31.611118+0000 mgr.smithi153.asomxr (mgr.14195) 654 : cluster 0 pgmap v543: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:35.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:34 smithi163 bash[22347]: cluster 2024-03-12T02:28:33.611744+0000 mgr.smithi153.asomxr (mgr.14195) 655 : cluster 0 pgmap v544: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:35.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:34 smithi153 bash[14340]: cluster 2024-03-12T02:28:33.611744+0000 mgr.smithi153.asomxr (mgr.14195) 655 : cluster 0 pgmap v544: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:36.512 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:28:37.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:36 smithi163 bash[22347]: cluster 2024-03-12T02:28:35.612368+0000 mgr.smithi153.asomxr (mgr.14195) 656 : cluster 0 pgmap v545: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:37.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:36 smithi153 bash[14340]: cluster 2024-03-12T02:28:35.612368+0000 mgr.smithi153.asomxr (mgr.14195) 656 : cluster 0 pgmap v545: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:39.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:38 smithi163 bash[22347]: cluster 2024-03-12T02:28:37.612860+0000 mgr.smithi153.asomxr (mgr.14195) 657 : cluster 0 pgmap v546: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:39.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:38 smithi153 bash[14340]: cluster 2024-03-12T02:28:37.612860+0000 mgr.smithi153.asomxr (mgr.14195) 657 : cluster 0 pgmap v546: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:39.851 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:28:39.851 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:28:40.642 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:28:41.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:40 smithi163 bash[22347]: cluster 2024-03-12T02:28:39.613523+0000 mgr.smithi153.asomxr (mgr.14195) 658 : cluster 0 pgmap v547: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:41.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:40 smithi163 bash[22347]: audit 2024-03-12T02:28:39.834939+0000 mgr.smithi153.asomxr (mgr.14195) 659 : audit 0 from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:41.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:40 smithi153 bash[14340]: cluster 2024-03-12T02:28:39.613523+0000 mgr.smithi153.asomxr (mgr.14195) 658 : cluster 0 pgmap v547: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:41.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:40 smithi153 bash[14340]: audit 2024-03-12T02:28:39.834939+0000 mgr.smithi153.asomxr (mgr.14195) 659 : audit 0 from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:41.643 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:28:43.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:42 smithi163 bash[22347]: cluster 2024-03-12T02:28:41.614106+0000 mgr.smithi153.asomxr (mgr.14195) 660 : cluster 0 pgmap v548: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:43.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:42 smithi153 bash[14340]: cluster 2024-03-12T02:28:41.614106+0000 mgr.smithi153.asomxr (mgr.14195) 660 : cluster 0 pgmap v548: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:45.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:44 smithi163 bash[22347]: cluster 2024-03-12T02:28:43.614710+0000 mgr.smithi153.asomxr (mgr.14195) 661 : cluster 0 pgmap v549: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:45.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:44 smithi153 bash[14340]: cluster 2024-03-12T02:28:43.614710+0000 mgr.smithi153.asomxr (mgr.14195) 661 : cluster 0 pgmap v549: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:46.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:45 smithi163 bash[22347]: audit 2024-03-12T02:28:45.402304+0000 mon.smithi153 (mon.0) 925 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:28:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:45 smithi153 bash[14340]: audit 2024-03-12T02:28:45.402304+0000 mon.smithi153 (mon.0) 925 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:28:46.390 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:28:47.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:46 smithi163 bash[22347]: cluster 2024-03-12T02:28:45.615281+0000 mgr.smithi153.asomxr (mgr.14195) 662 : cluster 0 pgmap v550: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:47.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:46 smithi153 bash[14340]: cluster 2024-03-12T02:28:45.615281+0000 mgr.smithi153.asomxr (mgr.14195) 662 : cluster 0 pgmap v550: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:49.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:48 smithi163 bash[22347]: cluster 2024-03-12T02:28:47.615846+0000 mgr.smithi153.asomxr (mgr.14195) 663 : cluster 0 pgmap v551: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:49.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:48 smithi153 bash[14340]: cluster 2024-03-12T02:28:47.615846+0000 mgr.smithi153.asomxr (mgr.14195) 663 : cluster 0 pgmap v551: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:49.761 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:28:49.761 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:28:50.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:49 smithi153 bash[14340]: audit 2024-03-12T02:28:49.720608+0000 mon.smithi153 (mon.0) 926 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:28:50.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:49 smithi163 bash[22347]: audit 2024-03-12T02:28:49.720608+0000 mon.smithi153 (mon.0) 926 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:28:50.592 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:28:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:50 smithi153 bash[14340]: cluster 2024-03-12T02:28:49.616445+0000 mgr.smithi153.asomxr (mgr.14195) 664 : cluster 0 pgmap v552: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:50 smithi153 bash[14340]: audit 2024-03-12T02:28:49.746489+0000 mgr.smithi153.asomxr (mgr.14195) 665 : audit 0 from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:50 smithi153 bash[14340]: audit 2024-03-12T02:28:50.248319+0000 mon.smithi153 (mon.0) 927 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:28:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:50 smithi153 bash[14340]: audit 2024-03-12T02:28:50.249679+0000 mon.smithi153 (mon.0) 928 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:28:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:50 smithi153 bash[14340]: audit 2024-03-12T02:28:50.256598+0000 mon.smithi153 (mon.0) 929 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:28:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:50 smithi153 bash[14340]: audit 2024-03-12T02:28:50.260179+0000 mon.smithi153 (mon.0) 930 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:28:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:50 smithi163 bash[22347]: cluster 2024-03-12T02:28:49.616445+0000 mgr.smithi153.asomxr (mgr.14195) 664 : cluster 0 pgmap v552: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:50 smithi163 bash[22347]: audit 2024-03-12T02:28:49.746489+0000 mgr.smithi153.asomxr (mgr.14195) 665 : audit 0 from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:28:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:50 smithi163 bash[22347]: audit 2024-03-12T02:28:50.248319+0000 mon.smithi153 (mon.0) 927 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:28:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:50 smithi163 bash[22347]: audit 2024-03-12T02:28:50.249679+0000 mon.smithi153 (mon.0) 928 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:28:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:50 smithi163 bash[22347]: audit 2024-03-12T02:28:50.256598+0000 mon.smithi153 (mon.0) 929 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:28:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:50 smithi163 bash[22347]: audit 2024-03-12T02:28:50.260179+0000 mon.smithi153 (mon.0) 930 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:28:51.593 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:28:52.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:51 smithi153 bash[14340]: cluster 2024-03-12T02:28:50.251965+0000 mgr.smithi153.asomxr (mgr.14195) 666 : cluster 0 pgmap v553: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:52.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:51 smithi153 bash[14340]: cluster 2024-03-12T02:28:50.252350+0000 mgr.smithi153.asomxr (mgr.14195) 667 : cluster 0 pgmap v554: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:51 smithi163 bash[22347]: cluster 2024-03-12T02:28:50.251965+0000 mgr.smithi153.asomxr (mgr.14195) 666 : cluster 0 pgmap v553: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:51 smithi163 bash[22347]: cluster 2024-03-12T02:28:50.252350+0000 mgr.smithi153.asomxr (mgr.14195) 667 : cluster 0 pgmap v554: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:54.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:53 smithi153 bash[14340]: cluster 2024-03-12T02:28:52.252935+0000 mgr.smithi153.asomxr (mgr.14195) 668 : cluster 0 pgmap v555: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:53 smithi163 bash[22347]: cluster 2024-03-12T02:28:52.252935+0000 mgr.smithi153.asomxr (mgr.14195) 668 : cluster 0 pgmap v555: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:56.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:55 smithi153 bash[14340]: cluster 2024-03-12T02:28:54.253572+0000 mgr.smithi153.asomxr (mgr.14195) 669 : cluster 0 pgmap v556: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:56.345 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:28:56.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:56 smithi163 bash[22347]: cluster 2024-03-12T02:28:54.253572+0000 mgr.smithi153.asomxr (mgr.14195) 669 : cluster 0 pgmap v556: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:58.298 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:28:58 smithi153 bash[14340]: cluster 2024-03-12T02:28:56.254232+0000 mgr.smithi153.asomxr (mgr.14195) 670 : cluster 0 pgmap v557: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:58.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:28:58 smithi163 bash[22347]: cluster 2024-03-12T02:28:56.254232+0000 mgr.smithi153.asomxr (mgr.14195) 670 : cluster 0 pgmap v557: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:28:59.723 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:28:59.723 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:29:00.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:00 smithi153 bash[14340]: cluster 2024-03-12T02:28:58.254788+0000 mgr.smithi153.asomxr (mgr.14195) 671 : cluster 0 pgmap v558: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:00.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:00 smithi163 bash[22347]: cluster 2024-03-12T02:28:58.254788+0000 mgr.smithi153.asomxr (mgr.14195) 671 : cluster 0 pgmap v558: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:00.553 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:29:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:01 smithi153 bash[14340]: audit 2024-03-12T02:28:59.707068+0000 mgr.smithi153.asomxr (mgr.14195) 672 : audit 0 from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:01.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:01 smithi153 bash[14340]: audit 2024-03-12T02:29:00.402126+0000 mon.smithi153 (mon.0) 931 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:29:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:01 smithi163 bash[22347]: audit 2024-03-12T02:28:59.707068+0000 mgr.smithi153.asomxr (mgr.14195) 672 : audit 0 from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:01 smithi163 bash[22347]: audit 2024-03-12T02:29:00.402126+0000 mon.smithi153 (mon.0) 931 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:29:01.554 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:29:02.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:02 smithi153 bash[14340]: cluster 2024-03-12T02:29:00.255407+0000 mgr.smithi153.asomxr (mgr.14195) 673 : cluster 0 pgmap v559: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:02 smithi163 bash[22347]: cluster 2024-03-12T02:29:00.255407+0000 mgr.smithi153.asomxr (mgr.14195) 673 : cluster 0 pgmap v559: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:04.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:04 smithi153 bash[14340]: cluster 2024-03-12T02:29:02.256045+0000 mgr.smithi153.asomxr (mgr.14195) 674 : cluster 0 pgmap v560: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:04 smithi163 bash[22347]: cluster 2024-03-12T02:29:02.256045+0000 mgr.smithi153.asomxr (mgr.14195) 674 : cluster 0 pgmap v560: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:06.307 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:29:06.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:06 smithi153 bash[14340]: cluster 2024-03-12T02:29:04.256643+0000 mgr.smithi153.asomxr (mgr.14195) 675 : cluster 0 pgmap v561: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:06.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:06 smithi163 bash[22347]: cluster 2024-03-12T02:29:04.256643+0000 mgr.smithi153.asomxr (mgr.14195) 675 : cluster 0 pgmap v561: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:08.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:08 smithi153 bash[14340]: cluster 2024-03-12T02:29:06.257277+0000 mgr.smithi153.asomxr (mgr.14195) 676 : cluster 0 pgmap v562: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:08.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:08 smithi163 bash[22347]: cluster 2024-03-12T02:29:06.257277+0000 mgr.smithi153.asomxr (mgr.14195) 676 : cluster 0 pgmap v562: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:09.818 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:29:09.818 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:29:10.311 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:10 smithi163 bash[22347]: cluster 2024-03-12T02:29:08.257850+0000 mgr.smithi153.asomxr (mgr.14195) 677 : cluster 0 pgmap v563: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:10.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:10 smithi153 bash[14340]: cluster 2024-03-12T02:29:08.257850+0000 mgr.smithi153.asomxr (mgr.14195) 677 : cluster 0 pgmap v563: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:10.638 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:29:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:11 smithi153 bash[14340]: audit 2024-03-12T02:29:09.806443+0000 mgr.smithi153.asomxr (mgr.14195) 678 : audit 0 from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:11 smithi163 bash[22347]: audit 2024-03-12T02:29:09.806443+0000 mgr.smithi153.asomxr (mgr.14195) 678 : audit 0 from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:11.639 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:29:12.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:12 smithi153 bash[14340]: cluster 2024-03-12T02:29:10.258489+0000 mgr.smithi153.asomxr (mgr.14195) 679 : cluster 0 pgmap v564: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:12.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:12 smithi163 bash[22347]: cluster 2024-03-12T02:29:10.258489+0000 mgr.smithi153.asomxr (mgr.14195) 679 : cluster 0 pgmap v564: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:14.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:14 smithi153 bash[14340]: cluster 2024-03-12T02:29:12.259155+0000 mgr.smithi153.asomxr (mgr.14195) 680 : cluster 0 pgmap v565: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:14.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:14 smithi163 bash[22347]: cluster 2024-03-12T02:29:12.259155+0000 mgr.smithi153.asomxr (mgr.14195) 680 : cluster 0 pgmap v565: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:16 smithi153 bash[14340]: cluster 2024-03-12T02:29:14.259760+0000 mgr.smithi153.asomxr (mgr.14195) 681 : cluster 0 pgmap v566: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:16 smithi153 bash[14340]: audit 2024-03-12T02:29:15.403138+0000 mon.smithi153 (mon.0) 932 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:29:16.395 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:29:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:16 smithi163 bash[22347]: cluster 2024-03-12T02:29:14.259760+0000 mgr.smithi153.asomxr (mgr.14195) 681 : cluster 0 pgmap v566: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:16 smithi163 bash[22347]: audit 2024-03-12T02:29:15.403138+0000 mon.smithi153 (mon.0) 932 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:29:18.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:18 smithi153 bash[14340]: cluster 2024-03-12T02:29:16.260370+0000 mgr.smithi153.asomxr (mgr.14195) 682 : cluster 0 pgmap v567: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:18.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:18 smithi163 bash[22347]: cluster 2024-03-12T02:29:16.260370+0000 mgr.smithi153.asomxr (mgr.14195) 682 : cluster 0 pgmap v567: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:19.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:19 smithi163 bash[22347]: cluster 2024-03-12T02:29:18.260773+0000 mgr.smithi153.asomxr (mgr.14195) 683 : cluster 0 pgmap v568: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:19 smithi153 bash[14340]: cluster 2024-03-12T02:29:18.260773+0000 mgr.smithi153.asomxr (mgr.14195) 683 : cluster 0 pgmap v568: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:19.753 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:29:19.753 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:29:20.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:20 smithi153 bash[14340]: audit 2024-03-12T02:29:19.741570+0000 mgr.smithi153.asomxr (mgr.14195) 684 : audit 0 from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:20.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:20 smithi163 bash[22347]: audit 2024-03-12T02:29:19.741570+0000 mgr.smithi153.asomxr (mgr.14195) 684 : audit 0 from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:20.550 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:29:21.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:21 smithi163 bash[22347]: cluster 2024-03-12T02:29:20.261293+0000 mgr.smithi153.asomxr (mgr.14195) 685 : cluster 0 pgmap v569: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:21.551 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:29:21.563 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:21 smithi153 bash[14340]: cluster 2024-03-12T02:29:20.261293+0000 mgr.smithi153.asomxr (mgr.14195) 685 : cluster 0 pgmap v569: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:24.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:23 smithi163 bash[22347]: cluster 2024-03-12T02:29:22.261860+0000 mgr.smithi153.asomxr (mgr.14195) 686 : cluster 0 pgmap v570: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:24.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:23 smithi153 bash[14340]: cluster 2024-03-12T02:29:22.261860+0000 mgr.smithi153.asomxr (mgr.14195) 686 : cluster 0 pgmap v570: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:26.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:25 smithi163 bash[22347]: cluster 2024-03-12T02:29:24.262472+0000 mgr.smithi153.asomxr (mgr.14195) 687 : cluster 0 pgmap v571: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:26.305 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:29:26.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:25 smithi153 bash[14340]: cluster 2024-03-12T02:29:24.262472+0000 mgr.smithi153.asomxr (mgr.14195) 687 : cluster 0 pgmap v571: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:27.969 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:27 smithi153 bash[14340]: cluster 2024-03-12T02:29:26.263115+0000 mgr.smithi153.asomxr (mgr.14195) 688 : cluster 0 pgmap v572: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:28.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:27 smithi163 bash[22347]: cluster 2024-03-12T02:29:26.263115+0000 mgr.smithi153.asomxr (mgr.14195) 688 : cluster 0 pgmap v572: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:29.486 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:29:29.486 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:29:30.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:29 smithi153 bash[14340]: cluster 2024-03-12T02:29:28.263690+0000 mgr.smithi153.asomxr (mgr.14195) 689 : cluster 0 pgmap v573: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:30.230 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:29:30.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:29 smithi163 bash[22347]: cluster 2024-03-12T02:29:28.263690+0000 mgr.smithi153.asomxr (mgr.14195) 689 : cluster 0 pgmap v573: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:30.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:30 smithi163 bash[22347]: audit 2024-03-12T02:29:29.470559+0000 mgr.smithi153.asomxr (mgr.14195) 690 : audit 0 from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:30.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:30 smithi163 bash[22347]: audit 2024-03-12T02:29:30.403330+0000 mon.smithi153 (mon.0) 933 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:29:31.230 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:29:31.244 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:30 smithi153 bash[14340]: audit 2024-03-12T02:29:29.470559+0000 mgr.smithi153.asomxr (mgr.14195) 690 : audit 0 from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:31.244 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:30 smithi153 bash[14340]: audit 2024-03-12T02:29:30.403330+0000 mon.smithi153 (mon.0) 933 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:29:32.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:31 smithi163 bash[22347]: cluster 2024-03-12T02:29:30.264303+0000 mgr.smithi153.asomxr (mgr.14195) 691 : cluster 0 pgmap v574: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:32.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:31 smithi153 bash[14340]: cluster 2024-03-12T02:29:30.264303+0000 mgr.smithi153.asomxr (mgr.14195) 691 : cluster 0 pgmap v574: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:34.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:33 smithi163 bash[22347]: cluster 2024-03-12T02:29:32.264915+0000 mgr.smithi153.asomxr (mgr.14195) 692 : cluster 0 pgmap v575: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:34.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:33 smithi153 bash[14340]: cluster 2024-03-12T02:29:32.264915+0000 mgr.smithi153.asomxr (mgr.14195) 692 : cluster 0 pgmap v575: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:35.989 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:29:36.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:35 smithi163 bash[22347]: cluster 2024-03-12T02:29:34.265608+0000 mgr.smithi153.asomxr (mgr.14195) 693 : cluster 0 pgmap v576: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:36.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:35 smithi153 bash[14340]: cluster 2024-03-12T02:29:34.265608+0000 mgr.smithi153.asomxr (mgr.14195) 693 : cluster 0 pgmap v576: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:38.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:37 smithi163 bash[22347]: cluster 2024-03-12T02:29:36.266274+0000 mgr.smithi153.asomxr (mgr.14195) 694 : cluster 0 pgmap v577: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:38.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:37 smithi153 bash[14340]: cluster 2024-03-12T02:29:36.266274+0000 mgr.smithi153.asomxr (mgr.14195) 694 : cluster 0 pgmap v577: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:39.464 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:29:39.464 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:29:40.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:39 smithi153 bash[14340]: cluster 2024-03-12T02:29:38.266897+0000 mgr.smithi153.asomxr (mgr.14195) 695 : cluster 0 pgmap v578: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:40.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:39 smithi163 bash[22347]: cluster 2024-03-12T02:29:38.266897+0000 mgr.smithi153.asomxr (mgr.14195) 695 : cluster 0 pgmap v578: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:40.277 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:29:41.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:40 smithi163 bash[22347]: audit 2024-03-12T02:29:39.452359+0000 mgr.smithi153.asomxr (mgr.14195) 696 : audit 0 from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:41.278 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:29:41.291 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:40 smithi153 bash[14340]: audit 2024-03-12T02:29:39.452359+0000 mgr.smithi153.asomxr (mgr.14195) 696 : audit 0 from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:42.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:41 smithi163 bash[22347]: cluster 2024-03-12T02:29:40.267570+0000 mgr.smithi153.asomxr (mgr.14195) 697 : cluster 0 pgmap v579: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:42.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:41 smithi153 bash[14340]: cluster 2024-03-12T02:29:40.267570+0000 mgr.smithi153.asomxr (mgr.14195) 697 : cluster 0 pgmap v579: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:44.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:43 smithi163 bash[22347]: cluster 2024-03-12T02:29:42.268177+0000 mgr.smithi153.asomxr (mgr.14195) 698 : cluster 0 pgmap v580: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:44.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:43 smithi153 bash[14340]: cluster 2024-03-12T02:29:42.268177+0000 mgr.smithi153.asomxr (mgr.14195) 698 : cluster 0 pgmap v580: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:46.031 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:29:46.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:45 smithi163 bash[22347]: cluster 2024-03-12T02:29:44.268875+0000 mgr.smithi153.asomxr (mgr.14195) 699 : cluster 0 pgmap v581: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:46.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:45 smithi163 bash[22347]: audit 2024-03-12T02:29:45.403912+0000 mon.smithi153 (mon.0) 934 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:29:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:45 smithi153 bash[14340]: cluster 2024-03-12T02:29:44.268875+0000 mgr.smithi153.asomxr (mgr.14195) 699 : cluster 0 pgmap v581: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:45 smithi153 bash[14340]: audit 2024-03-12T02:29:45.403912+0000 mon.smithi153 (mon.0) 934 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:29:48.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:47 smithi163 bash[22347]: cluster 2024-03-12T02:29:46.269516+0000 mgr.smithi153.asomxr (mgr.14195) 700 : cluster 0 pgmap v582: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:48.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:47 smithi153 bash[14340]: cluster 2024-03-12T02:29:46.269516+0000 mgr.smithi153.asomxr (mgr.14195) 700 : cluster 0 pgmap v582: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:49.371 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:29:49.372 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:29:50.129 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:29:50.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:49 smithi163 bash[22347]: cluster 2024-03-12T02:29:48.270125+0000 mgr.smithi153.asomxr (mgr.14195) 701 : cluster 0 pgmap v583: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:50.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:49 smithi153 bash[14340]: cluster 2024-03-12T02:29:48.270125+0000 mgr.smithi153.asomxr (mgr.14195) 701 : cluster 0 pgmap v583: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:51.130 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:29:51.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:50 smithi163 bash[22347]: audit 2024-03-12T02:29:49.354415+0000 mgr.smithi153.asomxr (mgr.14195) 702 : audit 0 from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:51.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:50 smithi163 bash[22347]: audit 2024-03-12T02:29:50.342497+0000 mon.smithi153 (mon.0) 935 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:29:51.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:50 smithi163 bash[22347]: audit 2024-03-12T02:29:50.874973+0000 mon.smithi153 (mon.0) 936 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:29:51.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:50 smithi163 bash[22347]: audit 2024-03-12T02:29:50.876364+0000 mon.smithi153 (mon.0) 937 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:29:51.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:50 smithi163 bash[22347]: audit 2024-03-12T02:29:50.885280+0000 mon.smithi153 (mon.0) 938 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:29:51.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:50 smithi163 bash[22347]: audit 2024-03-12T02:29:50.890175+0000 mon.smithi153 (mon.0) 939 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:29:51.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:50 smithi153 bash[14340]: audit 2024-03-12T02:29:49.354415+0000 mgr.smithi153.asomxr (mgr.14195) 702 : audit 0 from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:29:51.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:50 smithi153 bash[14340]: audit 2024-03-12T02:29:50.342497+0000 mon.smithi153 (mon.0) 935 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:29:51.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:50 smithi153 bash[14340]: audit 2024-03-12T02:29:50.874973+0000 mon.smithi153 (mon.0) 936 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:29:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:50 smithi153 bash[14340]: audit 2024-03-12T02:29:50.876364+0000 mon.smithi153 (mon.0) 937 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:29:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:50 smithi153 bash[14340]: audit 2024-03-12T02:29:50.885280+0000 mon.smithi153 (mon.0) 938 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:29:51.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:50 smithi153 bash[14340]: audit 2024-03-12T02:29:50.890175+0000 mon.smithi153 (mon.0) 939 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:29:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:51 smithi163 bash[22347]: cluster 2024-03-12T02:29:50.270749+0000 mgr.smithi153.asomxr (mgr.14195) 703 : cluster 0 pgmap v584: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:51 smithi163 bash[22347]: cluster 2024-03-12T02:29:50.878902+0000 mgr.smithi153.asomxr (mgr.14195) 704 : cluster 0 pgmap v585: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:51 smithi163 bash[22347]: cluster 2024-03-12T02:29:50.879295+0000 mgr.smithi153.asomxr (mgr.14195) 705 : cluster 0 pgmap v586: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:52.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:51 smithi153 bash[14340]: cluster 2024-03-12T02:29:50.270749+0000 mgr.smithi153.asomxr (mgr.14195) 703 : cluster 0 pgmap v584: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:52.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:51 smithi153 bash[14340]: cluster 2024-03-12T02:29:50.878902+0000 mgr.smithi153.asomxr (mgr.14195) 704 : cluster 0 pgmap v585: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:52.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:51 smithi153 bash[14340]: cluster 2024-03-12T02:29:50.879295+0000 mgr.smithi153.asomxr (mgr.14195) 705 : cluster 0 pgmap v586: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:54.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:53 smithi163 bash[22347]: cluster 2024-03-12T02:29:52.879972+0000 mgr.smithi153.asomxr (mgr.14195) 706 : cluster 0 pgmap v587: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:54.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:53 smithi153 bash[14340]: cluster 2024-03-12T02:29:52.879972+0000 mgr.smithi153.asomxr (mgr.14195) 706 : cluster 0 pgmap v587: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:55.883 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:29:56.203 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:55 smithi163 bash[22347]: cluster 2024-03-12T02:29:54.880590+0000 mgr.smithi153.asomxr (mgr.14195) 707 : cluster 0 pgmap v588: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:56.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:55 smithi153 bash[14340]: cluster 2024-03-12T02:29:54.880590+0000 mgr.smithi153.asomxr (mgr.14195) 707 : cluster 0 pgmap v588: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:58.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:57 smithi163 bash[22347]: cluster 2024-03-12T02:29:56.881262+0000 mgr.smithi153.asomxr (mgr.14195) 708 : cluster 0 pgmap v589: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:58.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:57 smithi153 bash[14340]: cluster 2024-03-12T02:29:56.881262+0000 mgr.smithi153.asomxr (mgr.14195) 708 : cluster 0 pgmap v589: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:29:59.240 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:29:59.240 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:30:00.055 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:30:00.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:29:59 smithi163 bash[22347]: cluster 2024-03-12T02:29:58.881854+0000 mgr.smithi153.asomxr (mgr.14195) 709 : cluster 0 pgmap v590: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:00.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:29:59 smithi153 bash[14340]: cluster 2024-03-12T02:29:58.881854+0000 mgr.smithi153.asomxr (mgr.14195) 709 : cluster 0 pgmap v590: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:01.056 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:30:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:00 smithi163 bash[22347]: audit 2024-03-12T02:29:59.221320+0000 mgr.smithi153.asomxr (mgr.14195) 710 : audit 0 from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:00 smithi163 bash[22347]: cluster 2024-03-12T02:30:00.000127+0000 mon.smithi153 (mon.0) 940 : cluster 3 overall HEALTH_WARN 2 failed cephadm daemon(s) 2024-03-12T02:30:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:00 smithi163 bash[22347]: audit 2024-03-12T02:30:00.404246+0000 mon.smithi153 (mon.0) 941 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:30:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:00 smithi153 bash[14340]: audit 2024-03-12T02:29:59.221320+0000 mgr.smithi153.asomxr (mgr.14195) 710 : audit 0 from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:00 smithi153 bash[14340]: cluster 2024-03-12T02:30:00.000127+0000 mon.smithi153 (mon.0) 940 : cluster 3 overall HEALTH_WARN 2 failed cephadm daemon(s) 2024-03-12T02:30:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:00 smithi153 bash[14340]: audit 2024-03-12T02:30:00.404246+0000 mon.smithi153 (mon.0) 941 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:30:02.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:01 smithi163 bash[22347]: cluster 2024-03-12T02:30:00.882478+0000 mgr.smithi153.asomxr (mgr.14195) 711 : cluster 0 pgmap v591: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:02.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:01 smithi153 bash[14340]: cluster 2024-03-12T02:30:00.882478+0000 mgr.smithi153.asomxr (mgr.14195) 711 : cluster 0 pgmap v591: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:04.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:03 smithi163 bash[22347]: cluster 2024-03-12T02:30:02.883112+0000 mgr.smithi153.asomxr (mgr.14195) 712 : cluster 0 pgmap v592: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:04.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:03 smithi153 bash[14340]: cluster 2024-03-12T02:30:02.883112+0000 mgr.smithi153.asomxr (mgr.14195) 712 : cluster 0 pgmap v592: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:05.808 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:30:06.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:05 smithi163 bash[22347]: cluster 2024-03-12T02:30:04.883714+0000 mgr.smithi153.asomxr (mgr.14195) 713 : cluster 0 pgmap v593: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:06.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:05 smithi153 bash[14340]: cluster 2024-03-12T02:30:04.883714+0000 mgr.smithi153.asomxr (mgr.14195) 713 : cluster 0 pgmap v593: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:08.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:07 smithi163 bash[22347]: cluster 2024-03-12T02:30:06.884324+0000 mgr.smithi153.asomxr (mgr.14195) 714 : cluster 0 pgmap v594: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:08.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:07 smithi153 bash[14340]: cluster 2024-03-12T02:30:06.884324+0000 mgr.smithi153.asomxr (mgr.14195) 714 : cluster 0 pgmap v594: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:09.119 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:30:09.120 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:30:09.904 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:30:10.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:09 smithi163 bash[22347]: cluster 2024-03-12T02:30:08.884820+0000 mgr.smithi153.asomxr (mgr.14195) 715 : cluster 0 pgmap v595: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:10.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:09 smithi153 bash[14340]: cluster 2024-03-12T02:30:08.884820+0000 mgr.smithi153.asomxr (mgr.14195) 715 : cluster 0 pgmap v595: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:10.905 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:30:11.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:10 smithi163 bash[22347]: audit 2024-03-12T02:30:09.103268+0000 mgr.smithi153.asomxr (mgr.14195) 716 : audit 0 from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:10 smithi153 bash[14340]: audit 2024-03-12T02:30:09.103268+0000 mgr.smithi153.asomxr (mgr.14195) 716 : audit 0 from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:12.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:11 smithi163 bash[22347]: cluster 2024-03-12T02:30:10.885453+0000 mgr.smithi153.asomxr (mgr.14195) 717 : cluster 0 pgmap v596: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:12.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:11 smithi153 bash[14340]: cluster 2024-03-12T02:30:10.885453+0000 mgr.smithi153.asomxr (mgr.14195) 717 : cluster 0 pgmap v596: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:14.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:13 smithi153 bash[14340]: cluster 2024-03-12T02:30:12.886117+0000 mgr.smithi153.asomxr (mgr.14195) 718 : cluster 0 pgmap v597: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:14.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:13 smithi163 bash[22347]: cluster 2024-03-12T02:30:12.886117+0000 mgr.smithi153.asomxr (mgr.14195) 718 : cluster 0 pgmap v597: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:15.659 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:30:16.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:15 smithi163 bash[22347]: cluster 2024-03-12T02:30:14.886721+0000 mgr.smithi153.asomxr (mgr.14195) 719 : cluster 0 pgmap v598: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:16.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:15 smithi163 bash[22347]: audit 2024-03-12T02:30:15.404874+0000 mon.smithi153 (mon.0) 942 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:30:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:15 smithi153 bash[14340]: cluster 2024-03-12T02:30:14.886721+0000 mgr.smithi153.asomxr (mgr.14195) 719 : cluster 0 pgmap v598: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:15 smithi153 bash[14340]: audit 2024-03-12T02:30:15.404874+0000 mon.smithi153 (mon.0) 942 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:30:18.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:17 smithi153 bash[14340]: cluster 2024-03-12T02:30:16.887281+0000 mgr.smithi153.asomxr (mgr.14195) 720 : cluster 0 pgmap v599: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:18.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:17 smithi163 bash[22347]: cluster 2024-03-12T02:30:16.887281+0000 mgr.smithi153.asomxr (mgr.14195) 720 : cluster 0 pgmap v599: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:18.929 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:30:18.929 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:30:19.681 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:30:20.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:19 smithi163 bash[22347]: cluster 2024-03-12T02:30:18.887884+0000 mgr.smithi153.asomxr (mgr.14195) 721 : cluster 0 pgmap v600: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:20.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:19 smithi163 bash[22347]: audit 2024-03-12T02:30:18.914286+0000 mgr.smithi153.asomxr (mgr.14195) 722 : audit 0 from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:20.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:19 smithi153 bash[14340]: cluster 2024-03-12T02:30:18.887884+0000 mgr.smithi153.asomxr (mgr.14195) 721 : cluster 0 pgmap v600: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:20.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:19 smithi153 bash[14340]: audit 2024-03-12T02:30:18.914286+0000 mgr.smithi153.asomxr (mgr.14195) 722 : audit 0 from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:20.683 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:30:21.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:21 smithi163 bash[22347]: cluster 2024-03-12T02:30:20.888506+0000 mgr.smithi153.asomxr (mgr.14195) 723 : cluster 0 pgmap v601: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:22.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:21 smithi153 bash[14340]: cluster 2024-03-12T02:30:20.888506+0000 mgr.smithi153.asomxr (mgr.14195) 723 : cluster 0 pgmap v601: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:24.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:23 smithi163 bash[22347]: cluster 2024-03-12T02:30:22.888997+0000 mgr.smithi153.asomxr (mgr.14195) 724 : cluster 0 pgmap v602: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:24.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:23 smithi153 bash[14340]: cluster 2024-03-12T02:30:22.888997+0000 mgr.smithi153.asomxr (mgr.14195) 724 : cluster 0 pgmap v602: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:25.435 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:30:26.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:25 smithi163 bash[22347]: cluster 2024-03-12T02:30:24.889587+0000 mgr.smithi153.asomxr (mgr.14195) 725 : cluster 0 pgmap v603: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:26.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:25 smithi153 bash[14340]: cluster 2024-03-12T02:30:24.889587+0000 mgr.smithi153.asomxr (mgr.14195) 725 : cluster 0 pgmap v603: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:28.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:27 smithi163 bash[22347]: cluster 2024-03-12T02:30:26.890117+0000 mgr.smithi153.asomxr (mgr.14195) 726 : cluster 0 pgmap v604: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:28.267 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:27 smithi153 bash[14340]: cluster 2024-03-12T02:30:26.890117+0000 mgr.smithi153.asomxr (mgr.14195) 726 : cluster 0 pgmap v604: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:28.679 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:30:28.679 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:30:29.421 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:30:30.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:29 smithi163 bash[22347]: audit 2024-03-12T02:30:28.666611+0000 mgr.smithi153.asomxr (mgr.14195) 727 : audit 0 from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:30.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:29 smithi163 bash[22347]: cluster 2024-03-12T02:30:28.890716+0000 mgr.smithi153.asomxr (mgr.14195) 728 : cluster 0 pgmap v605: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:30.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:29 smithi153 bash[14340]: audit 2024-03-12T02:30:28.666611+0000 mgr.smithi153.asomxr (mgr.14195) 727 : audit 0 from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:30.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:29 smithi153 bash[14340]: cluster 2024-03-12T02:30:28.890716+0000 mgr.smithi153.asomxr (mgr.14195) 728 : cluster 0 pgmap v605: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:30.422 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:30:31.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:30 smithi163 bash[22347]: audit 2024-03-12T02:30:30.405334+0000 mon.smithi153 (mon.0) 943 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:30:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:30 smithi153 bash[14340]: audit 2024-03-12T02:30:30.405334+0000 mon.smithi153 (mon.0) 943 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:30:32.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:31 smithi163 bash[22347]: cluster 2024-03-12T02:30:30.891354+0000 mgr.smithi153.asomxr (mgr.14195) 729 : cluster 0 pgmap v606: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:32.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:31 smithi153 bash[14340]: cluster 2024-03-12T02:30:30.891354+0000 mgr.smithi153.asomxr (mgr.14195) 729 : cluster 0 pgmap v606: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:34.177 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:33 smithi153 bash[14340]: cluster 2024-03-12T02:30:32.891997+0000 mgr.smithi153.asomxr (mgr.14195) 730 : cluster 0 pgmap v607: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:34.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:33 smithi163 bash[22347]: cluster 2024-03-12T02:30:32.891997+0000 mgr.smithi153.asomxr (mgr.14195) 730 : cluster 0 pgmap v607: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:35.179 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:30:36.231 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:35 smithi153 bash[14340]: cluster 2024-03-12T02:30:34.892644+0000 mgr.smithi153.asomxr (mgr.14195) 731 : cluster 0 pgmap v608: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:36.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:35 smithi163 bash[22347]: cluster 2024-03-12T02:30:34.892644+0000 mgr.smithi153.asomxr (mgr.14195) 731 : cluster 0 pgmap v608: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:38.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:37 smithi163 bash[22347]: cluster 2024-03-12T02:30:36.893340+0000 mgr.smithi153.asomxr (mgr.14195) 732 : cluster 0 pgmap v609: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:38.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:37 smithi153 bash[14340]: cluster 2024-03-12T02:30:36.893340+0000 mgr.smithi153.asomxr (mgr.14195) 732 : cluster 0 pgmap v609: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:38.565 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:30:38.565 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:30:39.344 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:30:40.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:39 smithi163 bash[22347]: audit 2024-03-12T02:30:38.548373+0000 mgr.smithi153.asomxr (mgr.14195) 733 : audit 0 from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:40.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:39 smithi163 bash[22347]: cluster 2024-03-12T02:30:38.893765+0000 mgr.smithi153.asomxr (mgr.14195) 734 : cluster 0 pgmap v610: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:40.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:39 smithi153 bash[14340]: audit 2024-03-12T02:30:38.548373+0000 mgr.smithi153.asomxr (mgr.14195) 733 : audit 0 from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:40.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:39 smithi153 bash[14340]: cluster 2024-03-12T02:30:38.893765+0000 mgr.smithi153.asomxr (mgr.14195) 734 : cluster 0 pgmap v610: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:40.346 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:30:42.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:41 smithi163 bash[22347]: cluster 2024-03-12T02:30:40.894370+0000 mgr.smithi153.asomxr (mgr.14195) 735 : cluster 0 pgmap v611: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:42.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:41 smithi153 bash[14340]: cluster 2024-03-12T02:30:40.894370+0000 mgr.smithi153.asomxr (mgr.14195) 735 : cluster 0 pgmap v611: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:44.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:43 smithi163 bash[22347]: cluster 2024-03-12T02:30:42.894943+0000 mgr.smithi153.asomxr (mgr.14195) 736 : cluster 0 pgmap v612: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:44.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:43 smithi153 bash[14340]: cluster 2024-03-12T02:30:42.894943+0000 mgr.smithi153.asomxr (mgr.14195) 736 : cluster 0 pgmap v612: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:45.100 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:30:46.212 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:45 smithi153 bash[14340]: cluster 2024-03-12T02:30:44.895576+0000 mgr.smithi153.asomxr (mgr.14195) 737 : cluster 0 pgmap v613: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:46.212 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:45 smithi153 bash[14340]: audit 2024-03-12T02:30:45.405652+0000 mon.smithi153 (mon.0) 944 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:30:46.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:45 smithi163 bash[22347]: cluster 2024-03-12T02:30:44.895576+0000 mgr.smithi153.asomxr (mgr.14195) 737 : cluster 0 pgmap v613: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:46.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:45 smithi163 bash[22347]: audit 2024-03-12T02:30:45.405652+0000 mon.smithi153 (mon.0) 944 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:30:48.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:47 smithi163 bash[22347]: cluster 2024-03-12T02:30:46.896204+0000 mgr.smithi153.asomxr (mgr.14195) 738 : cluster 0 pgmap v614: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:48.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:47 smithi153 bash[14340]: cluster 2024-03-12T02:30:46.896204+0000 mgr.smithi153.asomxr (mgr.14195) 738 : cluster 0 pgmap v614: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:48.366 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:30:48.366 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:20:32.385962Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:20:32.385152Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:20:32.385426Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:20:32.386183Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:20:32.387297Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:20:32.387487Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:20:32.384896Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:20:32.384615Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:20:32.385694Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:20:32.386595Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:20:32.386388Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:30:49.163 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:30:50.163 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:30:50.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:49 smithi163 bash[22347]: audit 2024-03-12T02:30:48.355512+0000 mgr.smithi153.asomxr (mgr.14195) 739 : audit 0 from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:50.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:49 smithi163 bash[22347]: cluster 2024-03-12T02:30:48.896808+0000 mgr.smithi153.asomxr (mgr.14195) 740 : cluster 0 pgmap v615: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:50.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:49 smithi153 bash[14340]: audit 2024-03-12T02:30:48.355512+0000 mgr.smithi153.asomxr (mgr.14195) 739 : audit 0 from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:30:50.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:49 smithi153 bash[14340]: cluster 2024-03-12T02:30:48.896808+0000 mgr.smithi153.asomxr (mgr.14195) 740 : cluster 0 pgmap v615: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:51 smithi163 bash[22347]: cluster 2024-03-12T02:30:50.897544+0000 mgr.smithi153.asomxr (mgr.14195) 741 : cluster 0 pgmap v616: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:51 smithi163 bash[22347]: audit 2024-03-12T02:30:50.980903+0000 mon.smithi153 (mon.0) 945 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:30:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:51 smithi163 bash[22347]: audit 2024-03-12T02:30:51.359623+0000 mon.smithi153 (mon.0) 946 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:30:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:51 smithi163 bash[22347]: audit 2024-03-12T02:30:51.368140+0000 mon.smithi153 (mon.0) 947 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:30:52.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:51 smithi153 bash[14340]: cluster 2024-03-12T02:30:50.897544+0000 mgr.smithi153.asomxr (mgr.14195) 741 : cluster 0 pgmap v616: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:52.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:51 smithi153 bash[14340]: audit 2024-03-12T02:30:50.980903+0000 mon.smithi153 (mon.0) 945 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:30:52.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:51 smithi153 bash[14340]: audit 2024-03-12T02:30:51.359623+0000 mon.smithi153 (mon.0) 946 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:30:52.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:51 smithi153 bash[14340]: audit 2024-03-12T02:30:51.368140+0000 mon.smithi153 (mon.0) 947 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:30:54.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:53 smithi163 bash[22347]: cluster 2024-03-12T02:30:52.898117+0000 mgr.smithi153.asomxr (mgr.14195) 742 : cluster 0 pgmap v617: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:54.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:53 smithi153 bash[14340]: cluster 2024-03-12T02:30:52.898117+0000 mgr.smithi153.asomxr (mgr.14195) 742 : cluster 0 pgmap v617: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:54.908 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:30:56.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:55 smithi163 bash[22347]: cluster 2024-03-12T02:30:54.898821+0000 mgr.smithi153.asomxr (mgr.14195) 743 : cluster 0 pgmap v618: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:56.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:55 smithi153 bash[14340]: cluster 2024-03-12T02:30:54.898821+0000 mgr.smithi153.asomxr (mgr.14195) 743 : cluster 0 pgmap v618: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:58.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:57 smithi163 bash[22347]: cluster 2024-03-12T02:30:56.899449+0000 mgr.smithi153.asomxr (mgr.14195) 744 : cluster 0 pgmap v619: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:58.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:57 smithi163 bash[22347]: audit 2024-03-12T02:30:57.281874+0000 mon.smithi153 (mon.0) 948 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:30:58.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:57 smithi163 bash[22347]: audit 2024-03-12T02:30:57.288035+0000 mon.smithi153 (mon.0) 949 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:30:58.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:57 smithi163 bash[22347]: audit 2024-03-12T02:30:57.760410+0000 mon.smithi153 (mon.0) 950 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:30:58.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:57 smithi163 bash[22347]: audit 2024-03-12T02:30:57.761464+0000 mon.smithi153 (mon.0) 951 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:30:58.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:57 smithi163 bash[22347]: audit 2024-03-12T02:30:57.768881+0000 mon.smithi153 (mon.0) 952 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:30:58.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:57 smithi163 bash[22347]: audit 2024-03-12T02:30:57.772611+0000 mon.smithi153 (mon.0) 953 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:30:58.264 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:30:58.264 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:30:58.296 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:57 smithi153 bash[14340]: cluster 2024-03-12T02:30:56.899449+0000 mgr.smithi153.asomxr (mgr.14195) 744 : cluster 0 pgmap v619: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:58.296 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:57 smithi153 bash[14340]: audit 2024-03-12T02:30:57.281874+0000 mon.smithi153 (mon.0) 948 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:30:58.296 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:57 smithi153 bash[14340]: audit 2024-03-12T02:30:57.288035+0000 mon.smithi153 (mon.0) 949 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:30:58.296 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:57 smithi153 bash[14340]: audit 2024-03-12T02:30:57.760410+0000 mon.smithi153 (mon.0) 950 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:30:58.296 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:57 smithi153 bash[14340]: audit 2024-03-12T02:30:57.761464+0000 mon.smithi153 (mon.0) 951 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:30:58.296 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:57 smithi153 bash[14340]: audit 2024-03-12T02:30:57.768881+0000 mon.smithi153 (mon.0) 952 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:30:58.296 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:57 smithi153 bash[14340]: audit 2024-03-12T02:30:57.772611+0000 mon.smithi153 (mon.0) 953 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:30:59.050 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:30:59.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:58 smithi163 bash[22347]: cluster 2024-03-12T02:30:57.763479+0000 mgr.smithi153.asomxr (mgr.14195) 745 : cluster 0 pgmap v620: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:59.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:58 smithi163 bash[22347]: cluster 2024-03-12T02:30:57.763789+0000 mgr.smithi153.asomxr (mgr.14195) 746 : cluster 0 pgmap v621: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:58 smithi153 bash[14340]: cluster 2024-03-12T02:30:57.763479+0000 mgr.smithi153.asomxr (mgr.14195) 745 : cluster 0 pgmap v620: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:30:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:58 smithi153 bash[14340]: cluster 2024-03-12T02:30:57.763789+0000 mgr.smithi153.asomxr (mgr.14195) 746 : cluster 0 pgmap v621: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:00.051 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:31:00.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:30:59 smithi163 bash[22347]: audit 2024-03-12T02:30:58.253091+0000 mgr.smithi153.asomxr (mgr.14195) 747 : audit 0 from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:00.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:30:59 smithi153 bash[14340]: audit 2024-03-12T02:30:58.253091+0000 mgr.smithi153.asomxr (mgr.14195) 747 : audit 0 from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:00 smithi163 bash[22347]: cluster 2024-03-12T02:30:59.764439+0000 mgr.smithi153.asomxr (mgr.14195) 748 : cluster 0 pgmap v622: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:00 smithi163 bash[22347]: audit 2024-03-12T02:31:00.406008+0000 mon.smithi153 (mon.0) 954 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:31:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:00 smithi153 bash[14340]: cluster 2024-03-12T02:30:59.764439+0000 mgr.smithi153.asomxr (mgr.14195) 748 : cluster 0 pgmap v622: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:00 smithi153 bash[14340]: audit 2024-03-12T02:31:00.406008+0000 mon.smithi153 (mon.0) 954 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:31:03.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:02 smithi163 bash[22347]: cluster 2024-03-12T02:31:01.765048+0000 mgr.smithi153.asomxr (mgr.14195) 749 : cluster 0 pgmap v623: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:03.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:02 smithi153 bash[14340]: cluster 2024-03-12T02:31:01.765048+0000 mgr.smithi153.asomxr (mgr.14195) 749 : cluster 0 pgmap v623: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:04.792 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:31:05.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:04 smithi153 bash[14340]: cluster 2024-03-12T02:31:03.765787+0000 mgr.smithi153.asomxr (mgr.14195) 750 : cluster 0 pgmap v624: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:05.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:04 smithi163 bash[22347]: cluster 2024-03-12T02:31:03.765787+0000 mgr.smithi153.asomxr (mgr.14195) 750 : cluster 0 pgmap v624: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:07.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:06 smithi153 bash[14340]: cluster 2024-03-12T02:31:05.766466+0000 mgr.smithi153.asomxr (mgr.14195) 751 : cluster 0 pgmap v625: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:06 smithi163 bash[22347]: cluster 2024-03-12T02:31:05.766466+0000 mgr.smithi153.asomxr (mgr.14195) 751 : cluster 0 pgmap v625: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:08.129 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:31:08.130 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:31:08.864 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:31:09.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:08 smithi153 bash[14340]: cluster 2024-03-12T02:31:07.767205+0000 mgr.smithi153.asomxr (mgr.14195) 752 : cluster 0 pgmap v626: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:08 smithi163 bash[22347]: cluster 2024-03-12T02:31:07.767205+0000 mgr.smithi153.asomxr (mgr.14195) 752 : cluster 0 pgmap v626: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:09.865 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:31:10.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:09 smithi153 bash[14340]: audit 2024-03-12T02:31:08.113894+0000 mgr.smithi153.asomxr (mgr.14195) 753 : audit 0 from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:10.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:09 smithi163 bash[22347]: audit 2024-03-12T02:31:08.113894+0000 mgr.smithi153.asomxr (mgr.14195) 753 : audit 0 from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:10 smithi153 bash[14340]: cluster 2024-03-12T02:31:09.767870+0000 mgr.smithi153.asomxr (mgr.14195) 754 : cluster 0 pgmap v627: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:10 smithi163 bash[22347]: cluster 2024-03-12T02:31:09.767870+0000 mgr.smithi153.asomxr (mgr.14195) 754 : cluster 0 pgmap v627: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:13.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:12 smithi153 bash[14340]: cluster 2024-03-12T02:31:11.768628+0000 mgr.smithi153.asomxr (mgr.14195) 755 : cluster 0 pgmap v628: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:13.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:13 smithi163 bash[22347]: cluster 2024-03-12T02:31:11.768628+0000 mgr.smithi153.asomxr (mgr.14195) 755 : cluster 0 pgmap v628: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:14.618 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:31:15.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:15 smithi153 bash[14340]: cluster 2024-03-12T02:31:13.769310+0000 mgr.smithi153.asomxr (mgr.14195) 756 : cluster 0 pgmap v629: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:15.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:15 smithi163 bash[22347]: cluster 2024-03-12T02:31:13.769310+0000 mgr.smithi153.asomxr (mgr.14195) 756 : cluster 0 pgmap v629: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:16 smithi153 bash[14340]: audit 2024-03-12T02:31:15.406487+0000 mon.smithi153 (mon.0) 955 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:31:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:16 smithi163 bash[22347]: audit 2024-03-12T02:31:15.406487+0000 mon.smithi153 (mon.0) 955 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:31:17.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:17 smithi153 bash[14340]: cluster 2024-03-12T02:31:15.769860+0000 mgr.smithi153.asomxr (mgr.14195) 757 : cluster 0 pgmap v630: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:17.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:17 smithi163 bash[22347]: cluster 2024-03-12T02:31:15.769860+0000 mgr.smithi153.asomxr (mgr.14195) 757 : cluster 0 pgmap v630: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:17.942 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:31:17.942 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:31:18.718 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:31:19.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:19 smithi153 bash[14340]: cluster 2024-03-12T02:31:17.770512+0000 mgr.smithi153.asomxr (mgr.14195) 758 : cluster 0 pgmap v631: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:19.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:19 smithi153 bash[14340]: audit 2024-03-12T02:31:17.925899+0000 mgr.smithi153.asomxr (mgr.14195) 759 : audit 0 from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:19 smithi163 bash[22347]: cluster 2024-03-12T02:31:17.770512+0000 mgr.smithi153.asomxr (mgr.14195) 758 : cluster 0 pgmap v631: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:19 smithi163 bash[22347]: audit 2024-03-12T02:31:17.925899+0000 mgr.smithi153.asomxr (mgr.14195) 759 : audit 0 from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:19.720 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:31:21.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:21 smithi153 bash[14340]: cluster 2024-03-12T02:31:19.771253+0000 mgr.smithi153.asomxr (mgr.14195) 760 : cluster 0 pgmap v632: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:21 smithi163 bash[22347]: cluster 2024-03-12T02:31:19.771253+0000 mgr.smithi153.asomxr (mgr.14195) 760 : cluster 0 pgmap v632: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:23.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:23 smithi163 bash[22347]: cluster 2024-03-12T02:31:21.771900+0000 mgr.smithi153.asomxr (mgr.14195) 761 : cluster 0 pgmap v633: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:23.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:23 smithi153 bash[14340]: cluster 2024-03-12T02:31:21.771900+0000 mgr.smithi153.asomxr (mgr.14195) 761 : cluster 0 pgmap v633: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:24.473 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:31:25.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:25 smithi153 bash[14340]: cluster 2024-03-12T02:31:23.772658+0000 mgr.smithi153.asomxr (mgr.14195) 762 : cluster 0 pgmap v634: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:25.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:25 smithi163 bash[22347]: cluster 2024-03-12T02:31:23.772658+0000 mgr.smithi153.asomxr (mgr.14195) 762 : cluster 0 pgmap v634: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:27.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:27 smithi153 bash[14340]: cluster 2024-03-12T02:31:25.773355+0000 mgr.smithi153.asomxr (mgr.14195) 763 : cluster 0 pgmap v635: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:27.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:27 smithi163 bash[22347]: cluster 2024-03-12T02:31:25.773355+0000 mgr.smithi153.asomxr (mgr.14195) 763 : cluster 0 pgmap v635: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:27.790 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:31:27.791 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:31:28.546 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:31:29.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:29 smithi153 bash[14340]: cluster 2024-03-12T02:31:27.773909+0000 mgr.smithi153.asomxr (mgr.14195) 764 : cluster 0 pgmap v636: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:29.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:29 smithi153 bash[14340]: audit 2024-03-12T02:31:27.775252+0000 mgr.smithi153.asomxr (mgr.14195) 765 : audit 0 from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:29 smithi163 bash[22347]: cluster 2024-03-12T02:31:27.773909+0000 mgr.smithi153.asomxr (mgr.14195) 764 : cluster 0 pgmap v636: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:29 smithi163 bash[22347]: audit 2024-03-12T02:31:27.775252+0000 mgr.smithi153.asomxr (mgr.14195) 765 : audit 0 from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:29.548 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:31:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:31 smithi153 bash[14340]: cluster 2024-03-12T02:31:29.774530+0000 mgr.smithi153.asomxr (mgr.14195) 766 : cluster 0 pgmap v637: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:31 smithi153 bash[14340]: audit 2024-03-12T02:31:30.406895+0000 mon.smithi153 (mon.0) 956 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:31:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:31 smithi163 bash[22347]: cluster 2024-03-12T02:31:29.774530+0000 mgr.smithi153.asomxr (mgr.14195) 766 : cluster 0 pgmap v637: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:31 smithi163 bash[22347]: audit 2024-03-12T02:31:30.406895+0000 mon.smithi153 (mon.0) 956 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:31:33.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:33 smithi153 bash[14340]: cluster 2024-03-12T02:31:31.775211+0000 mgr.smithi153.asomxr (mgr.14195) 767 : cluster 0 pgmap v638: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:33.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:33 smithi163 bash[22347]: cluster 2024-03-12T02:31:31.775211+0000 mgr.smithi153.asomxr (mgr.14195) 767 : cluster 0 pgmap v638: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:34.305 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:31:35.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:35 smithi153 bash[14340]: cluster 2024-03-12T02:31:33.775814+0000 mgr.smithi153.asomxr (mgr.14195) 768 : cluster 0 pgmap v639: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:35.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:35 smithi163 bash[22347]: cluster 2024-03-12T02:31:33.775814+0000 mgr.smithi153.asomxr (mgr.14195) 768 : cluster 0 pgmap v639: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:37.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:37 smithi153 bash[14340]: cluster 2024-03-12T02:31:35.776475+0000 mgr.smithi153.asomxr (mgr.14195) 769 : cluster 0 pgmap v640: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:37.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:37 smithi163 bash[22347]: cluster 2024-03-12T02:31:35.776475+0000 mgr.smithi153.asomxr (mgr.14195) 769 : cluster 0 pgmap v640: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:37.804 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:31:37.804 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:31:38.558 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:31:39.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:39 smithi153 bash[14340]: cluster 2024-03-12T02:31:37.777076+0000 mgr.smithi153.asomxr (mgr.14195) 770 : cluster 0 pgmap v641: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:39.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:39 smithi153 bash[14340]: audit 2024-03-12T02:31:37.787652+0000 mgr.smithi153.asomxr (mgr.14195) 771 : audit 0 from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:39.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:39 smithi163 bash[22347]: cluster 2024-03-12T02:31:37.777076+0000 mgr.smithi153.asomxr (mgr.14195) 770 : cluster 0 pgmap v641: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:39.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:39 smithi163 bash[22347]: audit 2024-03-12T02:31:37.787652+0000 mgr.smithi153.asomxr (mgr.14195) 771 : audit 0 from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:39.559 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:31:41.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:41 smithi153 bash[14340]: cluster 2024-03-12T02:31:39.777676+0000 mgr.smithi153.asomxr (mgr.14195) 772 : cluster 0 pgmap v642: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:41.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:41 smithi163 bash[22347]: cluster 2024-03-12T02:31:39.777676+0000 mgr.smithi153.asomxr (mgr.14195) 772 : cluster 0 pgmap v642: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:43.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:43 smithi153 bash[14340]: cluster 2024-03-12T02:31:41.778245+0000 mgr.smithi153.asomxr (mgr.14195) 773 : cluster 0 pgmap v643: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:43.406 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:43 smithi163 bash[22347]: cluster 2024-03-12T02:31:41.778245+0000 mgr.smithi153.asomxr (mgr.14195) 773 : cluster 0 pgmap v643: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:44.316 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:31:45.357 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:45 smithi153 bash[14340]: cluster 2024-03-12T02:31:43.778863+0000 mgr.smithi153.asomxr (mgr.14195) 774 : cluster 0 pgmap v644: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:45.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:45 smithi163 bash[22347]: cluster 2024-03-12T02:31:43.778863+0000 mgr.smithi153.asomxr (mgr.14195) 774 : cluster 0 pgmap v644: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:46 smithi163 bash[22347]: audit 2024-03-12T02:31:45.406735+0000 mon.smithi153 (mon.0) 957 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:31:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:46 smithi153 bash[14340]: audit 2024-03-12T02:31:45.406735+0000 mon.smithi153 (mon.0) 957 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:31:47.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:47 smithi153 bash[14340]: cluster 2024-03-12T02:31:45.779494+0000 mgr.smithi153.asomxr (mgr.14195) 775 : cluster 0 pgmap v645: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:47.450 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:31:47.451 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:31:47.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:47 smithi163 bash[22347]: cluster 2024-03-12T02:31:45.779494+0000 mgr.smithi153.asomxr (mgr.14195) 775 : cluster 0 pgmap v645: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:48.252 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:31:49.254 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:31:49.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:49 smithi163 bash[22347]: audit 2024-03-12T02:31:47.439663+0000 mgr.smithi153.asomxr (mgr.14195) 776 : audit 0 from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:49.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:49 smithi163 bash[22347]: cluster 2024-03-12T02:31:47.780148+0000 mgr.smithi153.asomxr (mgr.14195) 777 : cluster 0 pgmap v646: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:49.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:49 smithi153 bash[14340]: audit 2024-03-12T02:31:47.439663+0000 mgr.smithi153.asomxr (mgr.14195) 776 : audit 0 from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:49.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:49 smithi153 bash[14340]: cluster 2024-03-12T02:31:47.780148+0000 mgr.smithi153.asomxr (mgr.14195) 777 : cluster 0 pgmap v646: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:50 smithi163 bash[22347]: cluster 2024-03-12T02:31:49.780750+0000 mgr.smithi153.asomxr (mgr.14195) 778 : cluster 0 pgmap v647: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:50.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:50 smithi153 bash[14340]: cluster 2024-03-12T02:31:49.780750+0000 mgr.smithi153.asomxr (mgr.14195) 778 : cluster 0 pgmap v647: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:53.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:52 smithi163 bash[22347]: cluster 2024-03-12T02:31:51.781383+0000 mgr.smithi153.asomxr (mgr.14195) 779 : cluster 0 pgmap v648: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:53.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:52 smithi153 bash[14340]: cluster 2024-03-12T02:31:51.781383+0000 mgr.smithi153.asomxr (mgr.14195) 779 : cluster 0 pgmap v648: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:54.000 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:31:55.179 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:54 smithi153 bash[14340]: cluster 2024-03-12T02:31:53.781932+0000 mgr.smithi153.asomxr (mgr.14195) 780 : cluster 0 pgmap v649: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:55.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:54 smithi163 bash[22347]: cluster 2024-03-12T02:31:53.781932+0000 mgr.smithi153.asomxr (mgr.14195) 780 : cluster 0 pgmap v649: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:57.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:56 smithi163 bash[22347]: cluster 2024-03-12T02:31:55.782320+0000 mgr.smithi153.asomxr (mgr.14195) 781 : cluster 0 pgmap v650: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:57.313 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:31:57.314 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:31:57.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:56 smithi153 bash[14340]: cluster 2024-03-12T02:31:55.782320+0000 mgr.smithi153.asomxr (mgr.14195) 781 : cluster 0 pgmap v650: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:58.084 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:31:58.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:57 smithi163 bash[22347]: audit 2024-03-12T02:31:57.857489+0000 mon.smithi153 (mon.0) 958 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:31:58.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:57 smithi153 bash[14340]: audit 2024-03-12T02:31:57.857489+0000 mon.smithi153 (mon.0) 958 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:31:59.085 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:31:59.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:58 smithi163 bash[22347]: audit 2024-03-12T02:31:57.296490+0000 mgr.smithi153.asomxr (mgr.14195) 782 : audit 0 from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:59.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:58 smithi163 bash[22347]: cluster 2024-03-12T02:31:57.782633+0000 mgr.smithi153.asomxr (mgr.14195) 783 : cluster 0 pgmap v651: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:59.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:58 smithi163 bash[22347]: audit 2024-03-12T02:31:58.442613+0000 mon.smithi153 (mon.0) 959 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:31:59.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:58 smithi163 bash[22347]: audit 2024-03-12T02:31:58.444329+0000 mon.smithi153 (mon.0) 960 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:31:59.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:58 smithi163 bash[22347]: audit 2024-03-12T02:31:58.454204+0000 mon.smithi153 (mon.0) 961 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:31:59.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:58 smithi163 bash[22347]: audit 2024-03-12T02:31:58.458341+0000 mon.smithi153 (mon.0) 962 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:31:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:58 smithi153 bash[14340]: audit 2024-03-12T02:31:57.296490+0000 mgr.smithi153.asomxr (mgr.14195) 782 : audit 0 from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:31:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:58 smithi153 bash[14340]: cluster 2024-03-12T02:31:57.782633+0000 mgr.smithi153.asomxr (mgr.14195) 783 : cluster 0 pgmap v651: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:31:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:58 smithi153 bash[14340]: audit 2024-03-12T02:31:58.442613+0000 mon.smithi153 (mon.0) 959 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:31:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:58 smithi153 bash[14340]: audit 2024-03-12T02:31:58.444329+0000 mon.smithi153 (mon.0) 960 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:31:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:58 smithi153 bash[14340]: audit 2024-03-12T02:31:58.454204+0000 mon.smithi153 (mon.0) 961 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:31:59.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:58 smithi153 bash[14340]: audit 2024-03-12T02:31:58.458341+0000 mon.smithi153 (mon.0) 962 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:32:00.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:59 smithi163 bash[22347]: cluster 2024-03-12T02:31:58.446891+0000 mgr.smithi153.asomxr (mgr.14195) 784 : cluster 0 pgmap v652: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:00.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:31:59 smithi163 bash[22347]: cluster 2024-03-12T02:31:58.447447+0000 mgr.smithi153.asomxr (mgr.14195) 785 : cluster 0 pgmap v653: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:00.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:59 smithi153 bash[14340]: cluster 2024-03-12T02:31:58.446891+0000 mgr.smithi153.asomxr (mgr.14195) 784 : cluster 0 pgmap v652: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:00.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:31:59 smithi153 bash[14340]: cluster 2024-03-12T02:31:58.447447+0000 mgr.smithi153.asomxr (mgr.14195) 785 : cluster 0 pgmap v653: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:00 smithi163 bash[22347]: audit 2024-03-12T02:32:00.407596+0000 mon.smithi153 (mon.0) 963 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:32:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:00 smithi153 bash[14340]: audit 2024-03-12T02:32:00.407596+0000 mon.smithi153 (mon.0) 963 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:32:02.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:01 smithi163 bash[22347]: cluster 2024-03-12T02:32:00.447982+0000 mgr.smithi153.asomxr (mgr.14195) 786 : cluster 0 pgmap v654: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:02.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:01 smithi153 bash[14340]: cluster 2024-03-12T02:32:00.447982+0000 mgr.smithi153.asomxr (mgr.14195) 786 : cluster 0 pgmap v654: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:03.834 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:32:04.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:03 smithi163 bash[22347]: cluster 2024-03-12T02:32:02.448554+0000 mgr.smithi153.asomxr (mgr.14195) 787 : cluster 0 pgmap v655: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:04.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:03 smithi153 bash[14340]: cluster 2024-03-12T02:32:02.448554+0000 mgr.smithi153.asomxr (mgr.14195) 787 : cluster 0 pgmap v655: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:06.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:05 smithi163 bash[22347]: cluster 2024-03-12T02:32:04.449247+0000 mgr.smithi153.asomxr (mgr.14195) 788 : cluster 0 pgmap v656: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:06.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:05 smithi153 bash[14340]: cluster 2024-03-12T02:32:04.449247+0000 mgr.smithi153.asomxr (mgr.14195) 788 : cluster 0 pgmap v656: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:07.222 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:32:07.222 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:32:07.971 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:32:08.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:07 smithi163 bash[22347]: cluster 2024-03-12T02:32:06.449811+0000 mgr.smithi153.asomxr (mgr.14195) 789 : cluster 0 pgmap v657: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:08.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:07 smithi153 bash[14340]: cluster 2024-03-12T02:32:06.449811+0000 mgr.smithi153.asomxr (mgr.14195) 789 : cluster 0 pgmap v657: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:08.971 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:32:09.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:08 smithi163 bash[22347]: audit 2024-03-12T02:32:07.206391+0000 mgr.smithi153.asomxr (mgr.14195) 790 : audit 0 from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:09.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:08 smithi153 bash[14340]: audit 2024-03-12T02:32:07.206391+0000 mgr.smithi153.asomxr (mgr.14195) 790 : audit 0 from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:10.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:09 smithi163 bash[22347]: cluster 2024-03-12T02:32:08.450450+0000 mgr.smithi153.asomxr (mgr.14195) 791 : cluster 0 pgmap v658: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:10.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:09 smithi153 bash[14340]: cluster 2024-03-12T02:32:08.450450+0000 mgr.smithi153.asomxr (mgr.14195) 791 : cluster 0 pgmap v658: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:12.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:11 smithi163 bash[22347]: cluster 2024-03-12T02:32:10.451088+0000 mgr.smithi153.asomxr (mgr.14195) 792 : cluster 0 pgmap v659: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:12.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:11 smithi153 bash[14340]: cluster 2024-03-12T02:32:10.451088+0000 mgr.smithi153.asomxr (mgr.14195) 792 : cluster 0 pgmap v659: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:13.721 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:32:14.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:13 smithi153 bash[14340]: cluster 2024-03-12T02:32:12.451689+0000 mgr.smithi153.asomxr (mgr.14195) 793 : cluster 0 pgmap v660: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:14.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:13 smithi163 bash[22347]: cluster 2024-03-12T02:32:12.451689+0000 mgr.smithi153.asomxr (mgr.14195) 793 : cluster 0 pgmap v660: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:16.138 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:15 smithi153 bash[14340]: cluster 2024-03-12T02:32:14.452327+0000 mgr.smithi153.asomxr (mgr.14195) 794 : cluster 0 pgmap v661: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:16.138 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:15 smithi153 bash[14340]: audit 2024-03-12T02:32:15.407397+0000 mon.smithi153 (mon.0) 964 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:32:16.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:15 smithi163 bash[22347]: cluster 2024-03-12T02:32:14.452327+0000 mgr.smithi153.asomxr (mgr.14195) 794 : cluster 0 pgmap v661: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:16.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:15 smithi163 bash[22347]: audit 2024-03-12T02:32:15.407397+0000 mon.smithi153 (mon.0) 964 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:32:17.328 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:32:17.329 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:32:18.091 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:32:18.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:17 smithi163 bash[22347]: cluster 2024-03-12T02:32:16.452947+0000 mgr.smithi153.asomxr (mgr.14195) 795 : cluster 0 pgmap v662: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:18.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:17 smithi153 bash[14340]: cluster 2024-03-12T02:32:16.452947+0000 mgr.smithi153.asomxr (mgr.14195) 795 : cluster 0 pgmap v662: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:19.093 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:32:19.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:18 smithi153 bash[14340]: audit 2024-03-12T02:32:17.311332+0000 mgr.smithi153.asomxr (mgr.14195) 796 : audit 0 from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:18 smithi163 bash[22347]: audit 2024-03-12T02:32:17.311332+0000 mgr.smithi153.asomxr (mgr.14195) 796 : audit 0 from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:20.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:19 smithi153 bash[14340]: cluster 2024-03-12T02:32:18.453573+0000 mgr.smithi153.asomxr (mgr.14195) 797 : cluster 0 pgmap v663: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:19 smithi163 bash[22347]: cluster 2024-03-12T02:32:18.453573+0000 mgr.smithi153.asomxr (mgr.14195) 797 : cluster 0 pgmap v663: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:22.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:21 smithi153 bash[14340]: cluster 2024-03-12T02:32:20.454193+0000 mgr.smithi153.asomxr (mgr.14195) 798 : cluster 0 pgmap v664: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:22.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:21 smithi163 bash[22347]: cluster 2024-03-12T02:32:20.454193+0000 mgr.smithi153.asomxr (mgr.14195) 798 : cluster 0 pgmap v664: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:23.846 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:32:24.258 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:23 smithi163 bash[22347]: cluster 2024-03-12T02:32:22.454780+0000 mgr.smithi153.asomxr (mgr.14195) 799 : cluster 0 pgmap v665: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:24.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:23 smithi153 bash[14340]: cluster 2024-03-12T02:32:22.454780+0000 mgr.smithi153.asomxr (mgr.14195) 799 : cluster 0 pgmap v665: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:26.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:25 smithi153 bash[14340]: cluster 2024-03-12T02:32:24.455413+0000 mgr.smithi153.asomxr (mgr.14195) 800 : cluster 0 pgmap v666: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:26 smithi163 bash[22347]: cluster 2024-03-12T02:32:24.455413+0000 mgr.smithi153.asomxr (mgr.14195) 800 : cluster 0 pgmap v666: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:27.200 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:32:27.201 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:32:27.989 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:32:28.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:28 smithi153 bash[14340]: cluster 2024-03-12T02:32:26.456056+0000 mgr.smithi153.asomxr (mgr.14195) 801 : cluster 0 pgmap v667: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:28 smithi163 bash[22347]: cluster 2024-03-12T02:32:26.456056+0000 mgr.smithi153.asomxr (mgr.14195) 801 : cluster 0 pgmap v667: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:28.990 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:32:29.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:29 smithi153 bash[14340]: audit 2024-03-12T02:32:27.189488+0000 mgr.smithi153.asomxr (mgr.14195) 802 : audit 0 from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:29.366 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:29 smithi163 bash[22347]: audit 2024-03-12T02:32:27.189488+0000 mgr.smithi153.asomxr (mgr.14195) 802 : audit 0 from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:30.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:30 smithi153 bash[14340]: cluster 2024-03-12T02:32:28.456784+0000 mgr.smithi153.asomxr (mgr.14195) 803 : cluster 0 pgmap v668: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:30.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:30 smithi163 bash[22347]: cluster 2024-03-12T02:32:28.456784+0000 mgr.smithi153.asomxr (mgr.14195) 803 : cluster 0 pgmap v668: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:31 smithi153 bash[14340]: audit 2024-03-12T02:32:30.408417+0000 mon.smithi153 (mon.0) 965 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:32:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:31 smithi163 bash[22347]: audit 2024-03-12T02:32:30.408417+0000 mon.smithi153 (mon.0) 965 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:32:32.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:32 smithi153 bash[14340]: cluster 2024-03-12T02:32:30.457519+0000 mgr.smithi153.asomxr (mgr.14195) 804 : cluster 0 pgmap v669: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:32 smithi163 bash[22347]: cluster 2024-03-12T02:32:30.457519+0000 mgr.smithi153.asomxr (mgr.14195) 804 : cluster 0 pgmap v669: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:33.744 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:32:34.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:34 smithi153 bash[14340]: cluster 2024-03-12T02:32:32.458272+0000 mgr.smithi153.asomxr (mgr.14195) 805 : cluster 0 pgmap v670: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:34.474 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:34 smithi163 bash[22347]: cluster 2024-03-12T02:32:32.458272+0000 mgr.smithi153.asomxr (mgr.14195) 805 : cluster 0 pgmap v670: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:36.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:36 smithi153 bash[14340]: cluster 2024-03-12T02:32:34.458937+0000 mgr.smithi153.asomxr (mgr.14195) 806 : cluster 0 pgmap v671: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:36.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:36 smithi163 bash[22347]: cluster 2024-03-12T02:32:34.458937+0000 mgr.smithi153.asomxr (mgr.14195) 806 : cluster 0 pgmap v671: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:37.033 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:32:37.033 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:32:37.903 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:32:38.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:38 smithi153 bash[14340]: cluster 2024-03-12T02:32:36.459698+0000 mgr.smithi153.asomxr (mgr.14195) 807 : cluster 0 pgmap v672: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:38.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:38 smithi153 bash[14340]: audit 2024-03-12T02:32:37.016339+0000 mgr.smithi153.asomxr (mgr.14195) 808 : audit 0 from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:38.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:38 smithi163 bash[22347]: cluster 2024-03-12T02:32:36.459698+0000 mgr.smithi153.asomxr (mgr.14195) 807 : cluster 0 pgmap v672: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:38.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:38 smithi163 bash[22347]: audit 2024-03-12T02:32:37.016339+0000 mgr.smithi153.asomxr (mgr.14195) 808 : audit 0 from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:38.903 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:32:40.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:40 smithi153 bash[14340]: cluster 2024-03-12T02:32:38.460472+0000 mgr.smithi153.asomxr (mgr.14195) 809 : cluster 0 pgmap v673: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:40.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:40 smithi163 bash[22347]: cluster 2024-03-12T02:32:38.460472+0000 mgr.smithi153.asomxr (mgr.14195) 809 : cluster 0 pgmap v673: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:42.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:42 smithi153 bash[14340]: cluster 2024-03-12T02:32:40.461151+0000 mgr.smithi153.asomxr (mgr.14195) 810 : cluster 0 pgmap v674: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:42.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:42 smithi163 bash[22347]: cluster 2024-03-12T02:32:40.461151+0000 mgr.smithi153.asomxr (mgr.14195) 810 : cluster 0 pgmap v674: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:43.656 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:32:44.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:44 smithi153 bash[14340]: cluster 2024-03-12T02:32:42.461827+0000 mgr.smithi153.asomxr (mgr.14195) 811 : cluster 0 pgmap v675: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:44.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:44 smithi163 bash[22347]: cluster 2024-03-12T02:32:42.461827+0000 mgr.smithi153.asomxr (mgr.14195) 811 : cluster 0 pgmap v675: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:46 smithi153 bash[14340]: cluster 2024-03-12T02:32:44.462283+0000 mgr.smithi153.asomxr (mgr.14195) 812 : cluster 0 pgmap v676: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:46 smithi153 bash[14340]: audit 2024-03-12T02:32:45.408943+0000 mon.smithi153 (mon.0) 966 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:32:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:46 smithi163 bash[22347]: cluster 2024-03-12T02:32:44.462283+0000 mgr.smithi153.asomxr (mgr.14195) 812 : cluster 0 pgmap v676: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:46 smithi163 bash[22347]: audit 2024-03-12T02:32:45.408943+0000 mon.smithi153 (mon.0) 966 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:32:46.848 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:32:46.848 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:32:47.668 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:32:48.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:48 smithi153 bash[14340]: cluster 2024-03-12T02:32:46.462927+0000 mgr.smithi153.asomxr (mgr.14195) 813 : cluster 0 pgmap v677: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:48.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:48 smithi153 bash[14340]: audit 2024-03-12T02:32:46.832104+0000 mgr.smithi153.asomxr (mgr.14195) 814 : audit 0 from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:48.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:48 smithi163 bash[22347]: cluster 2024-03-12T02:32:46.462927+0000 mgr.smithi153.asomxr (mgr.14195) 813 : cluster 0 pgmap v677: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:48.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:48 smithi163 bash[22347]: audit 2024-03-12T02:32:46.832104+0000 mgr.smithi153.asomxr (mgr.14195) 814 : audit 0 from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:48.669 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:32:50.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:50 smithi153 bash[14340]: cluster 2024-03-12T02:32:48.463591+0000 mgr.smithi153.asomxr (mgr.14195) 815 : cluster 0 pgmap v678: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:50 smithi163 bash[22347]: cluster 2024-03-12T02:32:48.463591+0000 mgr.smithi153.asomxr (mgr.14195) 815 : cluster 0 pgmap v678: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:52.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:52 smithi153 bash[14340]: cluster 2024-03-12T02:32:50.464208+0000 mgr.smithi153.asomxr (mgr.14195) 816 : cluster 0 pgmap v679: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:52 smithi163 bash[22347]: cluster 2024-03-12T02:32:50.464208+0000 mgr.smithi153.asomxr (mgr.14195) 816 : cluster 0 pgmap v679: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:53.423 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:32:54.376 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:54 smithi153 bash[14340]: cluster 2024-03-12T02:32:52.464789+0000 mgr.smithi153.asomxr (mgr.14195) 817 : cluster 0 pgmap v680: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:54 smithi163 bash[22347]: cluster 2024-03-12T02:32:52.464789+0000 mgr.smithi153.asomxr (mgr.14195) 817 : cluster 0 pgmap v680: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:56.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:56 smithi163 bash[22347]: cluster 2024-03-12T02:32:54.465472+0000 mgr.smithi153.asomxr (mgr.14195) 818 : cluster 0 pgmap v681: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:56.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:56 smithi153 bash[14340]: cluster 2024-03-12T02:32:54.465472+0000 mgr.smithi153.asomxr (mgr.14195) 818 : cluster 0 pgmap v681: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:56.655 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:32:56.656 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:22:45.743903Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:22:45.744200Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:22:45.746217Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:22:45.745990Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:22:45.746447Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:22:45.744693Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:22:45.744921Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:22:45.744461Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:22:45.745149Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:32:57.422 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:32:58.424 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:32:58.438 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:58 smithi153 bash[14340]: cluster 2024-03-12T02:32:56.466106+0000 mgr.smithi153.asomxr (mgr.14195) 819 : cluster 0 pgmap v682: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:58.438 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:58 smithi153 bash[14340]: audit 2024-03-12T02:32:56.644698+0000 mgr.smithi153.asomxr (mgr.14195) 820 : audit 0 from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:58.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:58 smithi163 bash[22347]: cluster 2024-03-12T02:32:56.466106+0000 mgr.smithi153.asomxr (mgr.14195) 819 : cluster 0 pgmap v682: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:32:58.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:58 smithi163 bash[22347]: audit 2024-03-12T02:32:56.644698+0000 mgr.smithi153.asomxr (mgr.14195) 820 : audit 0 from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:32:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:59 smithi163 bash[22347]: audit 2024-03-12T02:32:58.553093+0000 mon.smithi153 (mon.0) 967 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:32:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:59 smithi163 bash[22347]: audit 2024-03-12T02:32:58.929687+0000 mon.smithi153 (mon.0) 968 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:32:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:32:59 smithi163 bash[22347]: audit 2024-03-12T02:32:58.939543+0000 mon.smithi153 (mon.0) 969 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:32:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:59 smithi153 bash[14340]: audit 2024-03-12T02:32:58.553093+0000 mon.smithi153 (mon.0) 967 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:32:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:59 smithi153 bash[14340]: audit 2024-03-12T02:32:58.929687+0000 mon.smithi153 (mon.0) 968 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:32:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:32:59 smithi153 bash[14340]: audit 2024-03-12T02:32:58.939543+0000 mon.smithi153 (mon.0) 969 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:33:00.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:00 smithi153 bash[14340]: cluster 2024-03-12T02:32:58.466752+0000 mgr.smithi153.asomxr (mgr.14195) 821 : cluster 0 pgmap v683: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:00.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:00 smithi163 bash[22347]: cluster 2024-03-12T02:32:58.466752+0000 mgr.smithi153.asomxr (mgr.14195) 821 : cluster 0 pgmap v683: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:01 smithi163 bash[22347]: audit 2024-03-12T02:33:00.409311+0000 mon.smithi153 (mon.0) 970 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:33:01.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:01 smithi153 bash[14340]: audit 2024-03-12T02:33:00.409311+0000 mon.smithi153 (mon.0) 970 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:33:02.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:02 smithi153 bash[14340]: cluster 2024-03-12T02:33:00.467335+0000 mgr.smithi153.asomxr (mgr.14195) 822 : cluster 0 pgmap v684: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:02 smithi163 bash[22347]: cluster 2024-03-12T02:33:00.467335+0000 mgr.smithi153.asomxr (mgr.14195) 822 : cluster 0 pgmap v684: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:03.175 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:33:04.198 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:03 smithi153 bash[14340]: cluster 2024-03-12T02:33:02.467902+0000 mgr.smithi153.asomxr (mgr.14195) 823 : cluster 0 pgmap v685: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:04.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:03 smithi163 bash[22347]: cluster 2024-03-12T02:33:02.467902+0000 mgr.smithi153.asomxr (mgr.14195) 823 : cluster 0 pgmap v685: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:06.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:05 smithi153 bash[14340]: cluster 2024-03-12T02:33:04.468584+0000 mgr.smithi153.asomxr (mgr.14195) 824 : cluster 0 pgmap v686: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:06.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:05 smithi153 bash[14340]: audit 2024-03-12T02:33:04.819131+0000 mon.smithi153 (mon.0) 971 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:33:06.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:05 smithi153 bash[14340]: audit 2024-03-12T02:33:04.827038+0000 mon.smithi153 (mon.0) 972 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:33:06.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:05 smithi153 bash[14340]: audit 2024-03-12T02:33:05.352399+0000 mon.smithi153 (mon.0) 973 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:33:06.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:05 smithi153 bash[14340]: audit 2024-03-12T02:33:05.353819+0000 mon.smithi153 (mon.0) 974 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:33:06.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:05 smithi153 bash[14340]: audit 2024-03-12T02:33:05.362357+0000 mon.smithi153 (mon.0) 975 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:33:06.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:05 smithi153 bash[14340]: audit 2024-03-12T02:33:05.365990+0000 mon.smithi153 (mon.0) 976 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:33:06.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:05 smithi163 bash[22347]: cluster 2024-03-12T02:33:04.468584+0000 mgr.smithi153.asomxr (mgr.14195) 824 : cluster 0 pgmap v686: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:06.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:05 smithi163 bash[22347]: audit 2024-03-12T02:33:04.819131+0000 mon.smithi153 (mon.0) 971 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:33:06.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:05 smithi163 bash[22347]: audit 2024-03-12T02:33:04.827038+0000 mon.smithi153 (mon.0) 972 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:33:06.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:05 smithi163 bash[22347]: audit 2024-03-12T02:33:05.352399+0000 mon.smithi153 (mon.0) 973 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:33:06.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:05 smithi163 bash[22347]: audit 2024-03-12T02:33:05.353819+0000 mon.smithi153 (mon.0) 974 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:33:06.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:05 smithi163 bash[22347]: audit 2024-03-12T02:33:05.362357+0000 mon.smithi153 (mon.0) 975 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:33:06.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:05 smithi163 bash[22347]: audit 2024-03-12T02:33:05.365990+0000 mon.smithi153 (mon.0) 976 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:33:06.436 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:33:06.437 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:33:07.208 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:06 smithi153 bash[14340]: cluster 2024-03-12T02:33:05.355917+0000 mgr.smithi153.asomxr (mgr.14195) 825 : cluster 0 pgmap v687: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:07.208 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:06 smithi153 bash[14340]: cluster 2024-03-12T02:33:05.356237+0000 mgr.smithi153.asomxr (mgr.14195) 826 : cluster 0 pgmap v688: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:07.209 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:33:07.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:06 smithi163 bash[22347]: cluster 2024-03-12T02:33:05.355917+0000 mgr.smithi153.asomxr (mgr.14195) 825 : cluster 0 pgmap v687: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:07.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:06 smithi163 bash[22347]: cluster 2024-03-12T02:33:05.356237+0000 mgr.smithi153.asomxr (mgr.14195) 826 : cluster 0 pgmap v688: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:08.210 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:33:08.222 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:07 smithi153 bash[14340]: audit 2024-03-12T02:33:06.421693+0000 mgr.smithi153.asomxr (mgr.14195) 827 : audit 0 from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:08.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:07 smithi163 bash[22347]: audit 2024-03-12T02:33:06.421693+0000 mgr.smithi153.asomxr (mgr.14195) 827 : audit 0 from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:09.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:08 smithi163 bash[22347]: cluster 2024-03-12T02:33:07.356888+0000 mgr.smithi153.asomxr (mgr.14195) 828 : cluster 0 pgmap v689: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:09.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:08 smithi153 bash[14340]: cluster 2024-03-12T02:33:07.356888+0000 mgr.smithi153.asomxr (mgr.14195) 828 : cluster 0 pgmap v689: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:11.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:10 smithi163 bash[22347]: cluster 2024-03-12T02:33:09.357645+0000 mgr.smithi153.asomxr (mgr.14195) 829 : cluster 0 pgmap v690: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:10 smithi153 bash[14340]: cluster 2024-03-12T02:33:09.357645+0000 mgr.smithi153.asomxr (mgr.14195) 829 : cluster 0 pgmap v690: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:12.955 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:33:13.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:12 smithi163 bash[22347]: cluster 2024-03-12T02:33:11.358327+0000 mgr.smithi153.asomxr (mgr.14195) 830 : cluster 0 pgmap v691: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:13.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:12 smithi153 bash[14340]: cluster 2024-03-12T02:33:11.358327+0000 mgr.smithi153.asomxr (mgr.14195) 830 : cluster 0 pgmap v691: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:15.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:14 smithi163 bash[22347]: cluster 2024-03-12T02:33:13.358966+0000 mgr.smithi153.asomxr (mgr.14195) 831 : cluster 0 pgmap v692: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:15.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:14 smithi153 bash[14340]: cluster 2024-03-12T02:33:13.358966+0000 mgr.smithi153.asomxr (mgr.14195) 831 : cluster 0 pgmap v692: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:16.141 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:33:16.142 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:33:16.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:15 smithi163 bash[22347]: audit 2024-03-12T02:33:15.409323+0000 mon.smithi153 (mon.0) 977 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:33:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:15 smithi153 bash[14340]: audit 2024-03-12T02:33:15.409323+0000 mon.smithi153 (mon.0) 977 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:33:16.977 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:33:17.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:16 smithi163 bash[22347]: cluster 2024-03-12T02:33:15.359611+0000 mgr.smithi153.asomxr (mgr.14195) 832 : cluster 0 pgmap v693: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:17.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:16 smithi153 bash[14340]: cluster 2024-03-12T02:33:15.359611+0000 mgr.smithi153.asomxr (mgr.14195) 832 : cluster 0 pgmap v693: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:17.979 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:33:18.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:17 smithi163 bash[22347]: audit 2024-03-12T02:33:16.125845+0000 mgr.smithi153.asomxr (mgr.14195) 833 : audit 0 from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:18.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:17 smithi153 bash[14340]: audit 2024-03-12T02:33:16.125845+0000 mgr.smithi153.asomxr (mgr.14195) 833 : audit 0 from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:19.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:18 smithi163 bash[22347]: cluster 2024-03-12T02:33:17.360158+0000 mgr.smithi153.asomxr (mgr.14195) 834 : cluster 0 pgmap v694: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:19.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:18 smithi153 bash[14340]: cluster 2024-03-12T02:33:17.360158+0000 mgr.smithi153.asomxr (mgr.14195) 834 : cluster 0 pgmap v694: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:21.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:20 smithi163 bash[22347]: cluster 2024-03-12T02:33:19.360794+0000 mgr.smithi153.asomxr (mgr.14195) 835 : cluster 0 pgmap v695: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:21.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:20 smithi153 bash[14340]: cluster 2024-03-12T02:33:19.360794+0000 mgr.smithi153.asomxr (mgr.14195) 835 : cluster 0 pgmap v695: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:22.727 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:33:23.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:22 smithi153 bash[14340]: cluster 2024-03-12T02:33:21.361526+0000 mgr.smithi153.asomxr (mgr.14195) 836 : cluster 0 pgmap v696: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:23.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:22 smithi163 bash[22347]: cluster 2024-03-12T02:33:21.361526+0000 mgr.smithi153.asomxr (mgr.14195) 836 : cluster 0 pgmap v696: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:25.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:24 smithi153 bash[14340]: cluster 2024-03-12T02:33:23.362161+0000 mgr.smithi153.asomxr (mgr.14195) 837 : cluster 0 pgmap v697: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:25.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:24 smithi163 bash[22347]: cluster 2024-03-12T02:33:23.362161+0000 mgr.smithi153.asomxr (mgr.14195) 837 : cluster 0 pgmap v697: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:26.165 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:33:26.165 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:33:26.953 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:33:27.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:26 smithi153 bash[14340]: cluster 2024-03-12T02:33:25.362771+0000 mgr.smithi153.asomxr (mgr.14195) 838 : cluster 0 pgmap v698: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:27.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:26 smithi163 bash[22347]: cluster 2024-03-12T02:33:25.362771+0000 mgr.smithi153.asomxr (mgr.14195) 838 : cluster 0 pgmap v698: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:27.954 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:33:28.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:27 smithi153 bash[14340]: audit 2024-03-12T02:33:26.150146+0000 mgr.smithi153.asomxr (mgr.14195) 839 : audit 0 from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:28 smithi163 bash[22347]: audit 2024-03-12T02:33:26.150146+0000 mgr.smithi153.asomxr (mgr.14195) 839 : audit 0 from='client.14878 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:29.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:28 smithi153 bash[14340]: cluster 2024-03-12T02:33:27.363335+0000 mgr.smithi153.asomxr (mgr.14195) 840 : cluster 0 pgmap v699: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:29 smithi163 bash[22347]: cluster 2024-03-12T02:33:27.363335+0000 mgr.smithi153.asomxr (mgr.14195) 840 : cluster 0 pgmap v699: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:31 smithi153 bash[14340]: cluster 2024-03-12T02:33:29.363995+0000 mgr.smithi153.asomxr (mgr.14195) 841 : cluster 0 pgmap v700: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:31 smithi153 bash[14340]: audit 2024-03-12T02:33:30.410077+0000 mon.smithi153 (mon.0) 978 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:33:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:31 smithi163 bash[22347]: cluster 2024-03-12T02:33:29.363995+0000 mgr.smithi153.asomxr (mgr.14195) 841 : cluster 0 pgmap v700: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:31 smithi163 bash[22347]: audit 2024-03-12T02:33:30.410077+0000 mon.smithi153 (mon.0) 978 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:33:32.707 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:33:33.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:33 smithi153 bash[14340]: cluster 2024-03-12T02:33:31.364645+0000 mgr.smithi153.asomxr (mgr.14195) 842 : cluster 0 pgmap v701: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:33.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:33 smithi163 bash[22347]: cluster 2024-03-12T02:33:31.364645+0000 mgr.smithi153.asomxr (mgr.14195) 842 : cluster 0 pgmap v701: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:35.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:35 smithi153 bash[14340]: cluster 2024-03-12T02:33:33.365028+0000 mgr.smithi153.asomxr (mgr.14195) 843 : cluster 0 pgmap v702: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:35.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:35 smithi163 bash[22347]: cluster 2024-03-12T02:33:33.365028+0000 mgr.smithi153.asomxr (mgr.14195) 843 : cluster 0 pgmap v702: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:35.842 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:33:35.842 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:33:36.647 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:33:37.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:37 smithi153 bash[14340]: cluster 2024-03-12T02:33:35.365656+0000 mgr.smithi153.asomxr (mgr.14195) 844 : cluster 0 pgmap v703: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:37.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:37 smithi153 bash[14340]: audit 2024-03-12T02:33:35.830675+0000 mgr.smithi153.asomxr (mgr.14195) 845 : audit 0 from='client.14882 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:37.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:37 smithi163 bash[22347]: cluster 2024-03-12T02:33:35.365656+0000 mgr.smithi153.asomxr (mgr.14195) 844 : cluster 0 pgmap v703: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:37.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:37 smithi163 bash[22347]: audit 2024-03-12T02:33:35.830675+0000 mgr.smithi153.asomxr (mgr.14195) 845 : audit 0 from='client.14882 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:37.648 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:33:39.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:39 smithi153 bash[14340]: cluster 2024-03-12T02:33:37.366278+0000 mgr.smithi153.asomxr (mgr.14195) 846 : cluster 0 pgmap v704: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:39.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:39 smithi163 bash[22347]: cluster 2024-03-12T02:33:37.366278+0000 mgr.smithi153.asomxr (mgr.14195) 846 : cluster 0 pgmap v704: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:41.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:41 smithi163 bash[22347]: cluster 2024-03-12T02:33:39.366877+0000 mgr.smithi153.asomxr (mgr.14195) 847 : cluster 0 pgmap v705: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:41.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:41 smithi153 bash[14340]: cluster 2024-03-12T02:33:39.366877+0000 mgr.smithi153.asomxr (mgr.14195) 847 : cluster 0 pgmap v705: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:42.402 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:33:43.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:43 smithi153 bash[14340]: cluster 2024-03-12T02:33:41.367494+0000 mgr.smithi153.asomxr (mgr.14195) 848 : cluster 0 pgmap v706: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:43.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:43 smithi163 bash[22347]: cluster 2024-03-12T02:33:41.367494+0000 mgr.smithi153.asomxr (mgr.14195) 848 : cluster 0 pgmap v706: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:45.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:45 smithi153 bash[14340]: cluster 2024-03-12T02:33:43.368080+0000 mgr.smithi153.asomxr (mgr.14195) 849 : cluster 0 pgmap v707: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:45.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:45 smithi163 bash[22347]: cluster 2024-03-12T02:33:43.368080+0000 mgr.smithi153.asomxr (mgr.14195) 849 : cluster 0 pgmap v707: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:45.587 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:33:45.587 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:33:46.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:46 smithi153 bash[14340]: audit 2024-03-12T02:33:45.410473+0000 mon.smithi153 (mon.0) 979 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:33:46.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:46 smithi163 bash[22347]: audit 2024-03-12T02:33:45.410473+0000 mon.smithi153 (mon.0) 979 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:33:46.335 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:33:47.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:47 smithi153 bash[14340]: cluster 2024-03-12T02:33:45.368851+0000 mgr.smithi153.asomxr (mgr.14195) 850 : cluster 0 pgmap v708: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:47.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:47 smithi153 bash[14340]: audit 2024-03-12T02:33:45.569335+0000 mgr.smithi153.asomxr (mgr.14195) 851 : audit 0 from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:47.336 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:33:47.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:47 smithi163 bash[22347]: cluster 2024-03-12T02:33:45.368851+0000 mgr.smithi153.asomxr (mgr.14195) 850 : cluster 0 pgmap v708: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:47.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:47 smithi163 bash[22347]: audit 2024-03-12T02:33:45.569335+0000 mgr.smithi153.asomxr (mgr.14195) 851 : audit 0 from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:49.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:49 smithi153 bash[14340]: cluster 2024-03-12T02:33:47.369559+0000 mgr.smithi153.asomxr (mgr.14195) 852 : cluster 0 pgmap v709: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:49.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:49 smithi163 bash[22347]: cluster 2024-03-12T02:33:47.369559+0000 mgr.smithi153.asomxr (mgr.14195) 852 : cluster 0 pgmap v709: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:51.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:51 smithi153 bash[14340]: cluster 2024-03-12T02:33:49.370197+0000 mgr.smithi153.asomxr (mgr.14195) 853 : cluster 0 pgmap v710: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:51 smithi163 bash[22347]: cluster 2024-03-12T02:33:49.370197+0000 mgr.smithi153.asomxr (mgr.14195) 853 : cluster 0 pgmap v710: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:52.091 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:33:53.240 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:53 smithi153 bash[14340]: cluster 2024-03-12T02:33:51.370757+0000 mgr.smithi153.asomxr (mgr.14195) 854 : cluster 0 pgmap v711: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:53.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:53 smithi163 bash[22347]: cluster 2024-03-12T02:33:51.370757+0000 mgr.smithi153.asomxr (mgr.14195) 854 : cluster 0 pgmap v711: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:55.285 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:33:55.286 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:33:55.317 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:55 smithi153 bash[14340]: cluster 2024-03-12T02:33:53.371383+0000 mgr.smithi153.asomxr (mgr.14195) 855 : cluster 0 pgmap v712: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:55.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:55 smithi163 bash[22347]: cluster 2024-03-12T02:33:53.371383+0000 mgr.smithi153.asomxr (mgr.14195) 855 : cluster 0 pgmap v712: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:55.989 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:33:56.990 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:33:57.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:57 smithi153 bash[14340]: audit 2024-03-12T02:33:55.270285+0000 mgr.smithi153.asomxr (mgr.14195) 856 : audit 0 from='client.14890 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:57.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:57 smithi153 bash[14340]: cluster 2024-03-12T02:33:55.372159+0000 mgr.smithi153.asomxr (mgr.14195) 857 : cluster 0 pgmap v713: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:57.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:57 smithi163 bash[22347]: audit 2024-03-12T02:33:55.270285+0000 mgr.smithi153.asomxr (mgr.14195) 856 : audit 0 from='client.14890 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:33:57.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:57 smithi163 bash[22347]: cluster 2024-03-12T02:33:55.372159+0000 mgr.smithi153.asomxr (mgr.14195) 857 : cluster 0 pgmap v713: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:33:59 smithi153 bash[14340]: cluster 2024-03-12T02:33:57.372856+0000 mgr.smithi153.asomxr (mgr.14195) 858 : cluster 0 pgmap v714: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:33:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:33:59 smithi163 bash[22347]: cluster 2024-03-12T02:33:57.372856+0000 mgr.smithi153.asomxr (mgr.14195) 858 : cluster 0 pgmap v714: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:01 smithi163 bash[22347]: cluster 2024-03-12T02:33:59.373619+0000 mgr.smithi153.asomxr (mgr.14195) 859 : cluster 0 pgmap v715: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:01 smithi163 bash[22347]: audit 2024-03-12T02:34:00.410081+0000 mon.smithi153 (mon.0) 980 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:34:01.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:01 smithi153 bash[14340]: cluster 2024-03-12T02:33:59.373619+0000 mgr.smithi153.asomxr (mgr.14195) 859 : cluster 0 pgmap v715: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:01.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:01 smithi153 bash[14340]: audit 2024-03-12T02:34:00.410081+0000 mon.smithi153 (mon.0) 980 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:34:01.738 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:34:03.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:03 smithi153 bash[14340]: cluster 2024-03-12T02:34:01.374191+0000 mgr.smithi153.asomxr (mgr.14195) 860 : cluster 0 pgmap v716: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:03.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:03 smithi163 bash[22347]: cluster 2024-03-12T02:34:01.374191+0000 mgr.smithi153.asomxr (mgr.14195) 860 : cluster 0 pgmap v716: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:05.093 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:34:05.093 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:34:05.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:05 smithi163 bash[22347]: cluster 2024-03-12T02:34:03.374746+0000 mgr.smithi153.asomxr (mgr.14195) 861 : cluster 0 pgmap v717: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:05.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:05 smithi153 bash[14340]: cluster 2024-03-12T02:34:03.374746+0000 mgr.smithi153.asomxr (mgr.14195) 861 : cluster 0 pgmap v717: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:05.837 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:34:06.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:06 smithi163 bash[22347]: audit 2024-03-12T02:34:05.077094+0000 mgr.smithi153.asomxr (mgr.14195) 862 : audit 0 from='client.14894 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:06.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:06 smithi163 bash[22347]: audit 2024-03-12T02:34:05.456332+0000 mon.smithi153 (mon.0) 981 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:34:06.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:06 smithi163 bash[22347]: audit 2024-03-12T02:34:06.032683+0000 mon.smithi153 (mon.0) 982 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:34:06.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:06 smithi163 bash[22347]: audit 2024-03-12T02:34:06.034389+0000 mon.smithi153 (mon.0) 983 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:34:06.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:06 smithi163 bash[22347]: audit 2024-03-12T02:34:06.045043+0000 mon.smithi153 (mon.0) 984 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:34:06.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:06 smithi163 bash[22347]: audit 2024-03-12T02:34:06.049695+0000 mon.smithi153 (mon.0) 985 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:34:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:06 smithi153 bash[14340]: audit 2024-03-12T02:34:05.077094+0000 mgr.smithi153.asomxr (mgr.14195) 862 : audit 0 from='client.14894 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:06 smithi153 bash[14340]: audit 2024-03-12T02:34:05.456332+0000 mon.smithi153 (mon.0) 981 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:34:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:06 smithi153 bash[14340]: audit 2024-03-12T02:34:06.032683+0000 mon.smithi153 (mon.0) 982 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:34:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:06 smithi153 bash[14340]: audit 2024-03-12T02:34:06.034389+0000 mon.smithi153 (mon.0) 983 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:34:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:06 smithi153 bash[14340]: audit 2024-03-12T02:34:06.045043+0000 mon.smithi153 (mon.0) 984 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:34:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:06 smithi153 bash[14340]: audit 2024-03-12T02:34:06.049695+0000 mon.smithi153 (mon.0) 985 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:34:06.838 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:34:07.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:07 smithi153 bash[14340]: cluster 2024-03-12T02:34:05.375535+0000 mgr.smithi153.asomxr (mgr.14195) 863 : cluster 0 pgmap v718: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:07.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:07 smithi153 bash[14340]: cluster 2024-03-12T02:34:06.037353+0000 mgr.smithi153.asomxr (mgr.14195) 864 : cluster 0 pgmap v719: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:07.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:07 smithi153 bash[14340]: cluster 2024-03-12T02:34:06.037877+0000 mgr.smithi153.asomxr (mgr.14195) 865 : cluster 0 pgmap v720: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:07 smithi163 bash[22347]: cluster 2024-03-12T02:34:05.375535+0000 mgr.smithi153.asomxr (mgr.14195) 863 : cluster 0 pgmap v718: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:07 smithi163 bash[22347]: cluster 2024-03-12T02:34:06.037353+0000 mgr.smithi153.asomxr (mgr.14195) 864 : cluster 0 pgmap v719: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:07 smithi163 bash[22347]: cluster 2024-03-12T02:34:06.037877+0000 mgr.smithi153.asomxr (mgr.14195) 865 : cluster 0 pgmap v720: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:09.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:08 smithi163 bash[22347]: cluster 2024-03-12T02:34:08.038554+0000 mgr.smithi153.asomxr (mgr.14195) 866 : cluster 0 pgmap v721: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:09.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:08 smithi153 bash[14340]: cluster 2024-03-12T02:34:08.038554+0000 mgr.smithi153.asomxr (mgr.14195) 866 : cluster 0 pgmap v721: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:11.431 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:11 smithi163 bash[22347]: cluster 2024-03-12T02:34:10.039147+0000 mgr.smithi153.asomxr (mgr.14195) 867 : cluster 0 pgmap v722: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:11.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:11 smithi153 bash[14340]: cluster 2024-03-12T02:34:10.039147+0000 mgr.smithi153.asomxr (mgr.14195) 867 : cluster 0 pgmap v722: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:11.593 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:34:13.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:13 smithi153 bash[14340]: cluster 2024-03-12T02:34:12.039748+0000 mgr.smithi153.asomxr (mgr.14195) 868 : cluster 0 pgmap v723: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:13.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:13 smithi163 bash[22347]: cluster 2024-03-12T02:34:12.039748+0000 mgr.smithi153.asomxr (mgr.14195) 868 : cluster 0 pgmap v723: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:14.880 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:34:14.880 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:34:15.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:15 smithi153 bash[14340]: cluster 2024-03-12T02:34:14.040325+0000 mgr.smithi153.asomxr (mgr.14195) 869 : cluster 0 pgmap v724: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:15.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:15 smithi163 bash[22347]: cluster 2024-03-12T02:34:14.040325+0000 mgr.smithi153.asomxr (mgr.14195) 869 : cluster 0 pgmap v724: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:15.617 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:34:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:16 smithi163 bash[22347]: audit 2024-03-12T02:34:14.868595+0000 mgr.smithi153.asomxr (mgr.14195) 870 : audit 0 from='client.14898 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:16 smithi163 bash[22347]: audit 2024-03-12T02:34:15.411060+0000 mon.smithi153 (mon.0) 986 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:34:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:16 smithi153 bash[14340]: audit 2024-03-12T02:34:14.868595+0000 mgr.smithi153.asomxr (mgr.14195) 870 : audit 0 from='client.14898 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:16 smithi153 bash[14340]: audit 2024-03-12T02:34:15.411060+0000 mon.smithi153 (mon.0) 986 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:34:16.619 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:34:17.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:17 smithi163 bash[22347]: cluster 2024-03-12T02:34:16.040959+0000 mgr.smithi153.asomxr (mgr.14195) 871 : cluster 0 pgmap v725: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:17.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:17 smithi153 bash[14340]: cluster 2024-03-12T02:34:16.040959+0000 mgr.smithi153.asomxr (mgr.14195) 871 : cluster 0 pgmap v725: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:19.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:18 smithi163 bash[22347]: cluster 2024-03-12T02:34:18.041599+0000 mgr.smithi153.asomxr (mgr.14195) 872 : cluster 0 pgmap v726: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:19.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:18 smithi153 bash[14340]: cluster 2024-03-12T02:34:18.041599+0000 mgr.smithi153.asomxr (mgr.14195) 872 : cluster 0 pgmap v726: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:21.374 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:34:21.400 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:21 smithi153 bash[14340]: cluster 2024-03-12T02:34:20.042287+0000 mgr.smithi153.asomxr (mgr.14195) 873 : cluster 0 pgmap v727: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:21 smithi163 bash[22347]: cluster 2024-03-12T02:34:20.042287+0000 mgr.smithi153.asomxr (mgr.14195) 873 : cluster 0 pgmap v727: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:23.406 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:23 smithi153 bash[14340]: cluster 2024-03-12T02:34:22.042907+0000 mgr.smithi153.asomxr (mgr.14195) 874 : cluster 0 pgmap v728: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:23.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:23 smithi163 bash[22347]: cluster 2024-03-12T02:34:22.042907+0000 mgr.smithi153.asomxr (mgr.14195) 874 : cluster 0 pgmap v728: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:24.904 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:34:24.904 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:34:25.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:25 smithi153 bash[14340]: cluster 2024-03-12T02:34:24.043506+0000 mgr.smithi153.asomxr (mgr.14195) 875 : cluster 0 pgmap v729: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:25.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:25 smithi163 bash[22347]: cluster 2024-03-12T02:34:24.043506+0000 mgr.smithi153.asomxr (mgr.14195) 875 : cluster 0 pgmap v729: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:25.708 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:34:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:26 smithi163 bash[22347]: audit 2024-03-12T02:34:24.893167+0000 mgr.smithi153.asomxr (mgr.14195) 876 : audit 0 from='client.14902 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:26 smithi153 bash[14340]: audit 2024-03-12T02:34:24.893167+0000 mgr.smithi153.asomxr (mgr.14195) 876 : audit 0 from='client.14902 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:26.710 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:34:27.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:27 smithi163 bash[22347]: cluster 2024-03-12T02:34:26.044127+0000 mgr.smithi153.asomxr (mgr.14195) 877 : cluster 0 pgmap v730: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:27.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:27 smithi153 bash[14340]: cluster 2024-03-12T02:34:26.044127+0000 mgr.smithi153.asomxr (mgr.14195) 877 : cluster 0 pgmap v730: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:29.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:28 smithi153 bash[14340]: cluster 2024-03-12T02:34:28.044785+0000 mgr.smithi153.asomxr (mgr.14195) 878 : cluster 0 pgmap v731: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:29.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:28 smithi163 bash[22347]: cluster 2024-03-12T02:34:28.044785+0000 mgr.smithi153.asomxr (mgr.14195) 878 : cluster 0 pgmap v731: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:31.460 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:34:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:31 smithi163 bash[22347]: cluster 2024-03-12T02:34:30.045449+0000 mgr.smithi153.asomxr (mgr.14195) 879 : cluster 0 pgmap v732: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:31.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:31 smithi163 bash[22347]: audit 2024-03-12T02:34:30.411509+0000 mon.smithi153 (mon.0) 987 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:34:31.487 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:31 smithi153 bash[14340]: cluster 2024-03-12T02:34:30.045449+0000 mgr.smithi153.asomxr (mgr.14195) 879 : cluster 0 pgmap v732: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:31.487 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:31 smithi153 bash[14340]: audit 2024-03-12T02:34:30.411509+0000 mon.smithi153 (mon.0) 987 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:34:33.388 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:33 smithi153 bash[14340]: cluster 2024-03-12T02:34:32.046092+0000 mgr.smithi153.asomxr (mgr.14195) 880 : cluster 0 pgmap v733: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:33.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:33 smithi163 bash[22347]: cluster 2024-03-12T02:34:32.046092+0000 mgr.smithi153.asomxr (mgr.14195) 880 : cluster 0 pgmap v733: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:34.860 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:34:34.860 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:34:35.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:35 smithi153 bash[14340]: cluster 2024-03-12T02:34:34.046675+0000 mgr.smithi153.asomxr (mgr.14195) 881 : cluster 0 pgmap v734: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:35.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:35 smithi163 bash[22347]: cluster 2024-03-12T02:34:34.046675+0000 mgr.smithi153.asomxr (mgr.14195) 881 : cluster 0 pgmap v734: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:35.650 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:34:36.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:36 smithi163 bash[22347]: audit 2024-03-12T02:34:34.847321+0000 mgr.smithi153.asomxr (mgr.14195) 882 : audit 0 from='client.14906 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:36.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:36 smithi153 bash[14340]: audit 2024-03-12T02:34:34.847321+0000 mgr.smithi153.asomxr (mgr.14195) 882 : audit 0 from='client.14906 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:36.651 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:34:37.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:37 smithi163 bash[22347]: cluster 2024-03-12T02:34:36.047312+0000 mgr.smithi153.asomxr (mgr.14195) 883 : cluster 0 pgmap v735: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:37.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:37 smithi153 bash[14340]: cluster 2024-03-12T02:34:36.047312+0000 mgr.smithi153.asomxr (mgr.14195) 883 : cluster 0 pgmap v735: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:39.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:38 smithi163 bash[22347]: cluster 2024-03-12T02:34:38.047931+0000 mgr.smithi153.asomxr (mgr.14195) 884 : cluster 0 pgmap v736: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:39.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:38 smithi153 bash[14340]: cluster 2024-03-12T02:34:38.047931+0000 mgr.smithi153.asomxr (mgr.14195) 884 : cluster 0 pgmap v736: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:41.407 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:34:41.434 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:41 smithi153 bash[14340]: cluster 2024-03-12T02:34:40.048544+0000 mgr.smithi153.asomxr (mgr.14195) 885 : cluster 0 pgmap v737: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:41.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:41 smithi163 bash[22347]: cluster 2024-03-12T02:34:40.048544+0000 mgr.smithi153.asomxr (mgr.14195) 885 : cluster 0 pgmap v737: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:43.386 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:43 smithi153 bash[14340]: cluster 2024-03-12T02:34:42.049104+0000 mgr.smithi153.asomxr (mgr.14195) 886 : cluster 0 pgmap v738: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:43.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:43 smithi163 bash[22347]: cluster 2024-03-12T02:34:42.049104+0000 mgr.smithi153.asomxr (mgr.14195) 886 : cluster 0 pgmap v738: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:44.732 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:34:44.732 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:34:45.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:45 smithi153 bash[14340]: cluster 2024-03-12T02:34:44.049676+0000 mgr.smithi153.asomxr (mgr.14195) 887 : cluster 0 pgmap v739: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:45.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:45 smithi163 bash[22347]: cluster 2024-03-12T02:34:44.049676+0000 mgr.smithi153.asomxr (mgr.14195) 887 : cluster 0 pgmap v739: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:45.592 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:34:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:46 smithi163 bash[22347]: audit 2024-03-12T02:34:44.716920+0000 mgr.smithi153.asomxr (mgr.14195) 888 : audit 0 from='client.14910 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:46.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:46 smithi163 bash[22347]: audit 2024-03-12T02:34:45.411912+0000 mon.smithi153 (mon.0) 988 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:34:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:46 smithi153 bash[14340]: audit 2024-03-12T02:34:44.716920+0000 mgr.smithi153.asomxr (mgr.14195) 888 : audit 0 from='client.14910 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:46 smithi153 bash[14340]: audit 2024-03-12T02:34:45.411912+0000 mon.smithi153 (mon.0) 988 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:34:46.593 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:34:47.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:47 smithi163 bash[22347]: cluster 2024-03-12T02:34:46.050254+0000 mgr.smithi153.asomxr (mgr.14195) 889 : cluster 0 pgmap v740: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:47.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:47 smithi153 bash[14340]: cluster 2024-03-12T02:34:46.050254+0000 mgr.smithi153.asomxr (mgr.14195) 889 : cluster 0 pgmap v740: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:49.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:48 smithi163 bash[22347]: cluster 2024-03-12T02:34:48.050921+0000 mgr.smithi153.asomxr (mgr.14195) 890 : cluster 0 pgmap v741: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:49.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:48 smithi153 bash[14340]: cluster 2024-03-12T02:34:48.050921+0000 mgr.smithi153.asomxr (mgr.14195) 890 : cluster 0 pgmap v741: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:51.344 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:34:51.371 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:51 smithi153 bash[14340]: cluster 2024-03-12T02:34:50.051565+0000 mgr.smithi153.asomxr (mgr.14195) 891 : cluster 0 pgmap v742: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:51 smithi163 bash[22347]: cluster 2024-03-12T02:34:50.051565+0000 mgr.smithi153.asomxr (mgr.14195) 891 : cluster 0 pgmap v742: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:53.445 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:53 smithi153 bash[14340]: cluster 2024-03-12T02:34:52.052008+0000 mgr.smithi153.asomxr (mgr.14195) 892 : cluster 0 pgmap v743: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:53.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:53 smithi163 bash[22347]: cluster 2024-03-12T02:34:52.052008+0000 mgr.smithi153.asomxr (mgr.14195) 892 : cluster 0 pgmap v743: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:54.805 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:34:54.806 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:34:55.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:55 smithi153 bash[14340]: cluster 2024-03-12T02:34:54.052580+0000 mgr.smithi153.asomxr (mgr.14195) 893 : cluster 0 pgmap v744: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:55.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:55 smithi163 bash[22347]: cluster 2024-03-12T02:34:54.052580+0000 mgr.smithi153.asomxr (mgr.14195) 893 : cluster 0 pgmap v744: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:55.595 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:34:56.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:56 smithi163 bash[22347]: audit 2024-03-12T02:34:54.789393+0000 mgr.smithi153.asomxr (mgr.14195) 894 : audit 0 from='client.14914 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:56.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:56 smithi153 bash[14340]: audit 2024-03-12T02:34:54.789393+0000 mgr.smithi153.asomxr (mgr.14195) 894 : audit 0 from='client.14914 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:34:56.596 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:34:57.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:57 smithi163 bash[22347]: cluster 2024-03-12T02:34:56.053291+0000 mgr.smithi153.asomxr (mgr.14195) 895 : cluster 0 pgmap v745: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:57.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:57 smithi153 bash[14340]: cluster 2024-03-12T02:34:56.053291+0000 mgr.smithi153.asomxr (mgr.14195) 895 : cluster 0 pgmap v745: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:59.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:34:58 smithi163 bash[22347]: cluster 2024-03-12T02:34:58.053892+0000 mgr.smithi153.asomxr (mgr.14195) 896 : cluster 0 pgmap v746: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:34:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:34:58 smithi153 bash[14340]: cluster 2024-03-12T02:34:58.053892+0000 mgr.smithi153.asomxr (mgr.14195) 896 : cluster 0 pgmap v746: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:01 smithi153 bash[14340]: cluster 2024-03-12T02:35:00.054481+0000 mgr.smithi153.asomxr (mgr.14195) 897 : cluster 0 pgmap v747: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:01.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:01 smithi153 bash[14340]: audit 2024-03-12T02:35:00.412163+0000 mon.smithi153 (mon.0) 989 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:35:01.346 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:35:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:01 smithi163 bash[22347]: cluster 2024-03-12T02:35:00.054481+0000 mgr.smithi153.asomxr (mgr.14195) 897 : cluster 0 pgmap v747: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:01 smithi163 bash[22347]: audit 2024-03-12T02:35:00.412163+0000 mon.smithi153 (mon.0) 989 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:35:03.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:03 smithi153 bash[14340]: cluster 2024-03-12T02:35:02.054949+0000 mgr.smithi153.asomxr (mgr.14195) 898 : cluster 0 pgmap v748: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:03.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:03 smithi163 bash[22347]: cluster 2024-03-12T02:35:02.054949+0000 mgr.smithi153.asomxr (mgr.14195) 898 : cluster 0 pgmap v748: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:04.431 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:35:04.431 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:35:05.233 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:35:05.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:05 smithi163 bash[22347]: cluster 2024-03-12T02:35:04.055530+0000 mgr.smithi153.asomxr (mgr.14195) 899 : cluster 0 pgmap v749: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:05.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:05 smithi153 bash[14340]: cluster 2024-03-12T02:35:04.055530+0000 mgr.smithi153.asomxr (mgr.14195) 899 : cluster 0 pgmap v749: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:06.234 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:35:06.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:06 smithi163 bash[22347]: audit 2024-03-12T02:35:04.414743+0000 mgr.smithi153.asomxr (mgr.14195) 900 : audit 0 from='client.14918 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:06 smithi153 bash[14340]: audit 2024-03-12T02:35:04.414743+0000 mgr.smithi153.asomxr (mgr.14195) 900 : audit 0 from='client.14918 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:07 smithi163 bash[22347]: cluster 2024-03-12T02:35:06.056171+0000 mgr.smithi153.asomxr (mgr.14195) 901 : cluster 0 pgmap v750: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:07 smithi163 bash[22347]: audit 2024-03-12T02:35:06.142966+0000 mon.smithi153 (mon.0) 990 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:35:07.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:07 smithi163 bash[22347]: audit 2024-03-12T02:35:06.717889+0000 mon.smithi153 (mon.0) 991 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:35:07.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:07 smithi163 bash[22347]: audit 2024-03-12T02:35:06.719677+0000 mon.smithi153 (mon.0) 992 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:35:07.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:07 smithi163 bash[22347]: audit 2024-03-12T02:35:06.729687+0000 mon.smithi153 (mon.0) 993 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:35:07.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:07 smithi163 bash[22347]: audit 2024-03-12T02:35:06.733843+0000 mon.smithi153 (mon.0) 994 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:35:07.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:07 smithi153 bash[14340]: cluster 2024-03-12T02:35:06.056171+0000 mgr.smithi153.asomxr (mgr.14195) 901 : cluster 0 pgmap v750: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:07.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:07 smithi153 bash[14340]: audit 2024-03-12T02:35:06.142966+0000 mon.smithi153 (mon.0) 990 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:35:07.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:07 smithi153 bash[14340]: audit 2024-03-12T02:35:06.717889+0000 mon.smithi153 (mon.0) 991 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:35:07.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:07 smithi153 bash[14340]: audit 2024-03-12T02:35:06.719677+0000 mon.smithi153 (mon.0) 992 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:35:07.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:07 smithi153 bash[14340]: audit 2024-03-12T02:35:06.729687+0000 mon.smithi153 (mon.0) 993 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:35:07.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:07 smithi153 bash[14340]: audit 2024-03-12T02:35:06.733843+0000 mon.smithi153 (mon.0) 994 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:35:08.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:08 smithi163 bash[22347]: cluster 2024-03-12T02:35:06.722570+0000 mgr.smithi153.asomxr (mgr.14195) 902 : cluster 0 pgmap v751: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:08.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:08 smithi163 bash[22347]: cluster 2024-03-12T02:35:06.723053+0000 mgr.smithi153.asomxr (mgr.14195) 903 : cluster 0 pgmap v752: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:08.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:08 smithi153 bash[14340]: cluster 2024-03-12T02:35:06.722570+0000 mgr.smithi153.asomxr (mgr.14195) 902 : cluster 0 pgmap v751: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:08.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:08 smithi153 bash[14340]: cluster 2024-03-12T02:35:06.723053+0000 mgr.smithi153.asomxr (mgr.14195) 903 : cluster 0 pgmap v752: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:09 smithi163 bash[22347]: cluster 2024-03-12T02:35:08.723640+0000 mgr.smithi153.asomxr (mgr.14195) 904 : cluster 0 pgmap v753: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:09.567 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:09 smithi153 bash[14340]: cluster 2024-03-12T02:35:08.723640+0000 mgr.smithi153.asomxr (mgr.14195) 904 : cluster 0 pgmap v753: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:10.991 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:35:12.035 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:11 smithi153 bash[14340]: cluster 2024-03-12T02:35:10.724290+0000 mgr.smithi153.asomxr (mgr.14195) 905 : cluster 0 pgmap v754: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:12.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:11 smithi163 bash[22347]: cluster 2024-03-12T02:35:10.724290+0000 mgr.smithi153.asomxr (mgr.14195) 905 : cluster 0 pgmap v754: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:14.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:13 smithi163 bash[22347]: cluster 2024-03-12T02:35:12.724904+0000 mgr.smithi153.asomxr (mgr.14195) 906 : cluster 0 pgmap v755: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:14.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:13 smithi153 bash[14340]: cluster 2024-03-12T02:35:12.724904+0000 mgr.smithi153.asomxr (mgr.14195) 906 : cluster 0 pgmap v755: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:14.393 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:35:14.393 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:35:15.151 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:35:16.152 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:35:16.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:15 smithi163 bash[22347]: audit 2024-03-12T02:35:14.375918+0000 mgr.smithi153.asomxr (mgr.14195) 907 : audit 0 from='client.14922 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:16.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:15 smithi163 bash[22347]: cluster 2024-03-12T02:35:14.725486+0000 mgr.smithi153.asomxr (mgr.14195) 908 : cluster 0 pgmap v756: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:16.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:15 smithi163 bash[22347]: audit 2024-03-12T02:35:15.412812+0000 mon.smithi153 (mon.0) 995 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:35:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:15 smithi153 bash[14340]: audit 2024-03-12T02:35:14.375918+0000 mgr.smithi153.asomxr (mgr.14195) 907 : audit 0 from='client.14922 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:15 smithi153 bash[14340]: cluster 2024-03-12T02:35:14.725486+0000 mgr.smithi153.asomxr (mgr.14195) 908 : cluster 0 pgmap v756: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:15 smithi153 bash[14340]: audit 2024-03-12T02:35:15.412812+0000 mon.smithi153 (mon.0) 995 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:35:18.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:17 smithi163 bash[22347]: cluster 2024-03-12T02:35:16.726175+0000 mgr.smithi153.asomxr (mgr.14195) 909 : cluster 0 pgmap v757: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:18.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:17 smithi153 bash[14340]: cluster 2024-03-12T02:35:16.726175+0000 mgr.smithi153.asomxr (mgr.14195) 909 : cluster 0 pgmap v757: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:20.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:19 smithi153 bash[14340]: cluster 2024-03-12T02:35:18.726956+0000 mgr.smithi153.asomxr (mgr.14195) 910 : cluster 0 pgmap v758: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:19 smithi163 bash[22347]: cluster 2024-03-12T02:35:18.726956+0000 mgr.smithi153.asomxr (mgr.14195) 910 : cluster 0 pgmap v758: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:20.904 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:35:22.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:21 smithi153 bash[14340]: cluster 2024-03-12T02:35:20.727691+0000 mgr.smithi153.asomxr (mgr.14195) 911 : cluster 0 pgmap v759: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:22.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:21 smithi163 bash[22347]: cluster 2024-03-12T02:35:20.727691+0000 mgr.smithi153.asomxr (mgr.14195) 911 : cluster 0 pgmap v759: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:24.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:23 smithi153 bash[14340]: cluster 2024-03-12T02:35:22.728431+0000 mgr.smithi153.asomxr (mgr.14195) 912 : cluster 0 pgmap v760: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:24.358 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:35:24.358 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:35:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:23 smithi163 bash[22347]: cluster 2024-03-12T02:35:22.728431+0000 mgr.smithi153.asomxr (mgr.14195) 912 : cluster 0 pgmap v760: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:25.202 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:35:26.203 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:35:26.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:25 smithi153 bash[14340]: audit 2024-03-12T02:35:24.342017+0000 mgr.smithi153.asomxr (mgr.14195) 913 : audit 0 from='client.14926 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:26.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:25 smithi153 bash[14340]: cluster 2024-03-12T02:35:24.729048+0000 mgr.smithi153.asomxr (mgr.14195) 914 : cluster 0 pgmap v761: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:25 smithi163 bash[22347]: audit 2024-03-12T02:35:24.342017+0000 mgr.smithi153.asomxr (mgr.14195) 913 : audit 0 from='client.14926 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:25 smithi163 bash[22347]: cluster 2024-03-12T02:35:24.729048+0000 mgr.smithi153.asomxr (mgr.14195) 914 : cluster 0 pgmap v761: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:28.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:27 smithi163 bash[22347]: cluster 2024-03-12T02:35:26.729617+0000 mgr.smithi153.asomxr (mgr.14195) 915 : cluster 0 pgmap v762: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:28.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:27 smithi153 bash[14340]: cluster 2024-03-12T02:35:26.729617+0000 mgr.smithi153.asomxr (mgr.14195) 915 : cluster 0 pgmap v762: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:30.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:29 smithi153 bash[14340]: cluster 2024-03-12T02:35:28.730296+0000 mgr.smithi153.asomxr (mgr.14195) 916 : cluster 0 pgmap v763: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:30.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:30 smithi163 bash[22347]: cluster 2024-03-12T02:35:28.730296+0000 mgr.smithi153.asomxr (mgr.14195) 916 : cluster 0 pgmap v763: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:30.953 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:35:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:31 smithi153 bash[14340]: audit 2024-03-12T02:35:30.413233+0000 mon.smithi153 (mon.0) 996 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:35:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:31 smithi163 bash[22347]: audit 2024-03-12T02:35:30.413233+0000 mon.smithi153 (mon.0) 996 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:35:32.326 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:32 smithi153 bash[14340]: cluster 2024-03-12T02:35:30.730945+0000 mgr.smithi153.asomxr (mgr.14195) 917 : cluster 0 pgmap v764: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:32 smithi163 bash[22347]: cluster 2024-03-12T02:35:30.730945+0000 mgr.smithi153.asomxr (mgr.14195) 917 : cluster 0 pgmap v764: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:34.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:34 smithi153 bash[14340]: cluster 2024-03-12T02:35:32.731444+0000 mgr.smithi153.asomxr (mgr.14195) 918 : cluster 0 pgmap v765: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:34.417 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:35:34.418 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:35:34.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:34 smithi163 bash[22347]: cluster 2024-03-12T02:35:32.731444+0000 mgr.smithi153.asomxr (mgr.14195) 918 : cluster 0 pgmap v765: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:35.217 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:35:36.218 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:35:36.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:36 smithi153 bash[14340]: audit 2024-03-12T02:35:34.400918+0000 mgr.smithi153.asomxr (mgr.14195) 919 : audit 0 from='client.14930 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:36.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:36 smithi153 bash[14340]: cluster 2024-03-12T02:35:34.732021+0000 mgr.smithi153.asomxr (mgr.14195) 920 : cluster 0 pgmap v766: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:36.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:36 smithi163 bash[22347]: audit 2024-03-12T02:35:34.400918+0000 mgr.smithi153.asomxr (mgr.14195) 919 : audit 0 from='client.14930 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:36.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:36 smithi163 bash[22347]: cluster 2024-03-12T02:35:34.732021+0000 mgr.smithi153.asomxr (mgr.14195) 920 : cluster 0 pgmap v766: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:38.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:38 smithi153 bash[14340]: cluster 2024-03-12T02:35:36.732695+0000 mgr.smithi153.asomxr (mgr.14195) 921 : cluster 0 pgmap v767: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:38.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:38 smithi163 bash[22347]: cluster 2024-03-12T02:35:36.732695+0000 mgr.smithi153.asomxr (mgr.14195) 921 : cluster 0 pgmap v767: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:40.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:40 smithi153 bash[14340]: cluster 2024-03-12T02:35:38.733375+0000 mgr.smithi153.asomxr (mgr.14195) 922 : cluster 0 pgmap v768: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:40.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:40 smithi163 bash[22347]: cluster 2024-03-12T02:35:38.733375+0000 mgr.smithi153.asomxr (mgr.14195) 922 : cluster 0 pgmap v768: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:40.980 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:35:42.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:42 smithi153 bash[14340]: cluster 2024-03-12T02:35:40.734028+0000 mgr.smithi153.asomxr (mgr.14195) 923 : cluster 0 pgmap v769: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:42.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:42 smithi163 bash[22347]: cluster 2024-03-12T02:35:40.734028+0000 mgr.smithi153.asomxr (mgr.14195) 923 : cluster 0 pgmap v769: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:44.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:44 smithi153 bash[14340]: cluster 2024-03-12T02:35:42.734640+0000 mgr.smithi153.asomxr (mgr.14195) 924 : cluster 0 pgmap v770: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:44.239 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:35:44.239 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:35:44.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:44 smithi163 bash[22347]: cluster 2024-03-12T02:35:42.734640+0000 mgr.smithi153.asomxr (mgr.14195) 924 : cluster 0 pgmap v770: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:45.004 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:35:46.006 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:35:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:46 smithi153 bash[14340]: audit 2024-03-12T02:35:44.227511+0000 mgr.smithi153.asomxr (mgr.14195) 925 : audit 0 from='client.14934 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:46.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:46 smithi153 bash[14340]: cluster 2024-03-12T02:35:44.735195+0000 mgr.smithi153.asomxr (mgr.14195) 926 : cluster 0 pgmap v771: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:46.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:46 smithi153 bash[14340]: audit 2024-03-12T02:35:45.413888+0000 mon.smithi153 (mon.0) 997 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:35:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:46 smithi163 bash[22347]: audit 2024-03-12T02:35:44.227511+0000 mgr.smithi153.asomxr (mgr.14195) 925 : audit 0 from='client.14934 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:46 smithi163 bash[22347]: cluster 2024-03-12T02:35:44.735195+0000 mgr.smithi153.asomxr (mgr.14195) 926 : cluster 0 pgmap v771: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:46.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:46 smithi163 bash[22347]: audit 2024-03-12T02:35:45.413888+0000 mon.smithi153 (mon.0) 997 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:35:48.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:48 smithi153 bash[14340]: cluster 2024-03-12T02:35:46.735761+0000 mgr.smithi153.asomxr (mgr.14195) 927 : cluster 0 pgmap v772: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:48.326 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:48 smithi163 bash[22347]: cluster 2024-03-12T02:35:46.735761+0000 mgr.smithi153.asomxr (mgr.14195) 927 : cluster 0 pgmap v772: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:50.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:50 smithi153 bash[14340]: cluster 2024-03-12T02:35:48.736383+0000 mgr.smithi153.asomxr (mgr.14195) 928 : cluster 0 pgmap v773: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:50 smithi163 bash[22347]: cluster 2024-03-12T02:35:48.736383+0000 mgr.smithi153.asomxr (mgr.14195) 928 : cluster 0 pgmap v773: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:50.763 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:35:52.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:52 smithi153 bash[14340]: cluster 2024-03-12T02:35:50.737006+0000 mgr.smithi153.asomxr (mgr.14195) 929 : cluster 0 pgmap v774: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:52 smithi163 bash[22347]: cluster 2024-03-12T02:35:50.737006+0000 mgr.smithi153.asomxr (mgr.14195) 929 : cluster 0 pgmap v774: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:53.900 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:35:53.900 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:35:54.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:54 smithi153 bash[14340]: cluster 2024-03-12T02:35:52.737748+0000 mgr.smithi153.asomxr (mgr.14195) 930 : cluster 0 pgmap v775: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:54 smithi163 bash[22347]: cluster 2024-03-12T02:35:52.737748+0000 mgr.smithi153.asomxr (mgr.14195) 930 : cluster 0 pgmap v775: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:54.700 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:35:55.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:55 smithi153 bash[14340]: audit 2024-03-12T02:35:53.884140+0000 mgr.smithi153.asomxr (mgr.14195) 931 : audit 0 from='client.14938 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:55.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:55 smithi163 bash[22347]: audit 2024-03-12T02:35:53.884140+0000 mgr.smithi153.asomxr (mgr.14195) 931 : audit 0 from='client.14938 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:35:55.701 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:35:56.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:56 smithi153 bash[14340]: cluster 2024-03-12T02:35:54.738450+0000 mgr.smithi153.asomxr (mgr.14195) 932 : cluster 0 pgmap v776: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:56.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:56 smithi163 bash[22347]: cluster 2024-03-12T02:35:54.738450+0000 mgr.smithi153.asomxr (mgr.14195) 932 : cluster 0 pgmap v776: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:58.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:35:58 smithi163 bash[22347]: cluster 2024-03-12T02:35:56.739081+0000 mgr.smithi153.asomxr (mgr.14195) 933 : cluster 0 pgmap v777: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:35:58.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:35:58 smithi153 bash[14340]: cluster 2024-03-12T02:35:56.739081+0000 mgr.smithi153.asomxr (mgr.14195) 933 : cluster 0 pgmap v777: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:00.460 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:36:00.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:00 smithi163 bash[22347]: cluster 2024-03-12T02:35:58.739690+0000 mgr.smithi153.asomxr (mgr.14195) 934 : cluster 0 pgmap v778: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:00.489 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:00 smithi153 bash[14340]: cluster 2024-03-12T02:35:58.739690+0000 mgr.smithi153.asomxr (mgr.14195) 934 : cluster 0 pgmap v778: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:01 smithi163 bash[22347]: audit 2024-03-12T02:36:00.413726+0000 mon.smithi153 (mon.0) 998 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:36:01.566 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:01 smithi153 bash[14340]: audit 2024-03-12T02:36:00.413726+0000 mon.smithi153 (mon.0) 998 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:36:02.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:02 smithi153 bash[14340]: cluster 2024-03-12T02:36:00.740324+0000 mgr.smithi153.asomxr (mgr.14195) 935 : cluster 0 pgmap v779: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:02 smithi163 bash[22347]: cluster 2024-03-12T02:36:00.740324+0000 mgr.smithi153.asomxr (mgr.14195) 935 : cluster 0 pgmap v779: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:03.982 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:36:03.982 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:36:04.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:04 smithi153 bash[14340]: cluster 2024-03-12T02:36:02.740862+0000 mgr.smithi153.asomxr (mgr.14195) 936 : cluster 0 pgmap v780: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:04 smithi163 bash[22347]: cluster 2024-03-12T02:36:02.740862+0000 mgr.smithi153.asomxr (mgr.14195) 936 : cluster 0 pgmap v780: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:04.803 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:36:05.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:05 smithi163 bash[22347]: audit 2024-03-12T02:36:03.967156+0000 mgr.smithi153.asomxr (mgr.14195) 937 : audit 0 from='client.14942 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:05.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:05 smithi153 bash[14340]: audit 2024-03-12T02:36:03.967156+0000 mgr.smithi153.asomxr (mgr.14195) 937 : audit 0 from='client.14942 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:05.804 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:36:06.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:06 smithi163 bash[22347]: cluster 2024-03-12T02:36:04.741466+0000 mgr.smithi153.asomxr (mgr.14195) 938 : cluster 0 pgmap v781: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:06 smithi153 bash[14340]: cluster 2024-03-12T02:36:04.741466+0000 mgr.smithi153.asomxr (mgr.14195) 938 : cluster 0 pgmap v781: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:07 smithi163 bash[22347]: audit 2024-03-12T02:36:06.828165+0000 mon.smithi153 (mon.0) 999 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:36:07.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:07 smithi153 bash[14340]: audit 2024-03-12T02:36:06.828165+0000 mon.smithi153 (mon.0) 999 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:36:08.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:08 smithi163 bash[22347]: cluster 2024-03-12T02:36:06.742124+0000 mgr.smithi153.asomxr (mgr.14195) 939 : cluster 0 pgmap v782: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:08.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:08 smithi163 bash[22347]: audit 2024-03-12T02:36:07.419487+0000 mon.smithi153 (mon.0) 1000 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:36:08.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:08 smithi163 bash[22347]: audit 2024-03-12T02:36:07.421077+0000 mon.smithi153 (mon.0) 1001 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:36:08.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:08 smithi163 bash[22347]: audit 2024-03-12T02:36:07.422474+0000 mon.smithi153 (mon.0) 1002 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:36:08.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:08 smithi163 bash[22347]: audit 2024-03-12T02:36:07.423688+0000 mon.smithi153 (mon.0) 1003 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:36:08.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:08 smithi163 bash[22347]: audit 2024-03-12T02:36:07.431603+0000 mon.smithi153 (mon.0) 1004 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:36:08.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:08 smithi163 bash[22347]: audit 2024-03-12T02:36:07.432791+0000 mon.smithi153 (mon.0) 1005 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:36:08.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:08 smithi163 bash[22347]: audit 2024-03-12T02:36:07.433821+0000 mon.smithi153 (mon.0) 1006 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:36:08.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:08 smithi163 bash[22347]: audit 2024-03-12T02:36:07.439816+0000 mon.smithi153 (mon.0) 1007 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:36:08.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:08 smithi163 bash[22347]: audit 2024-03-12T02:36:07.445148+0000 mon.smithi153 (mon.0) 1008 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:36:08.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:08 smithi153 bash[14340]: cluster 2024-03-12T02:36:06.742124+0000 mgr.smithi153.asomxr (mgr.14195) 939 : cluster 0 pgmap v782: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:08.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:08 smithi153 bash[14340]: audit 2024-03-12T02:36:07.419487+0000 mon.smithi153 (mon.0) 1000 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:36:08.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:08 smithi153 bash[14340]: audit 2024-03-12T02:36:07.421077+0000 mon.smithi153 (mon.0) 1001 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:36:08.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:08 smithi153 bash[14340]: audit 2024-03-12T02:36:07.422474+0000 mon.smithi153 (mon.0) 1002 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:36:08.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:08 smithi153 bash[14340]: audit 2024-03-12T02:36:07.423688+0000 mon.smithi153 (mon.0) 1003 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:36:08.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:08 smithi153 bash[14340]: audit 2024-03-12T02:36:07.431603+0000 mon.smithi153 (mon.0) 1004 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:36:08.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:08 smithi153 bash[14340]: audit 2024-03-12T02:36:07.432791+0000 mon.smithi153 (mon.0) 1005 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:36:08.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:08 smithi153 bash[14340]: audit 2024-03-12T02:36:07.433821+0000 mon.smithi153 (mon.0) 1006 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:36:08.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:08 smithi153 bash[14340]: audit 2024-03-12T02:36:07.439816+0000 mon.smithi153 (mon.0) 1007 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:36:08.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:08 smithi153 bash[14340]: audit 2024-03-12T02:36:07.445148+0000 mon.smithi153 (mon.0) 1008 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:36:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:09 smithi163 bash[22347]: cephadm 2024-03-12T02:36:07.424700+0000 mgr.smithi153.asomxr (mgr.14195) 940 : cephadm 1 Adjusting osd_memory_target on smithi163 to 2930M 2024-03-12T02:36:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:09 smithi163 bash[22347]: cluster 2024-03-12T02:36:07.435742+0000 mgr.smithi153.asomxr (mgr.14195) 941 : cluster 0 pgmap v783: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:09 smithi163 bash[22347]: cluster 2024-03-12T02:36:07.435953+0000 mgr.smithi153.asomxr (mgr.14195) 942 : cluster 0 pgmap v784: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:09.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:09 smithi153 bash[14340]: cephadm 2024-03-12T02:36:07.424700+0000 mgr.smithi153.asomxr (mgr.14195) 940 : cephadm 1 Adjusting osd_memory_target on smithi163 to 2930M 2024-03-12T02:36:09.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:09 smithi153 bash[14340]: cluster 2024-03-12T02:36:07.435742+0000 mgr.smithi153.asomxr (mgr.14195) 941 : cluster 0 pgmap v783: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:09.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:09 smithi153 bash[14340]: cluster 2024-03-12T02:36:07.435953+0000 mgr.smithi153.asomxr (mgr.14195) 942 : cluster 0 pgmap v784: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:10.559 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:36:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:11 smithi163 bash[22347]: cluster 2024-03-12T02:36:09.436525+0000 mgr.smithi153.asomxr (mgr.14195) 943 : cluster 0 pgmap v785: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:11.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:11 smithi153 bash[14340]: cluster 2024-03-12T02:36:09.436525+0000 mgr.smithi153.asomxr (mgr.14195) 943 : cluster 0 pgmap v785: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:13.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:13 smithi153 bash[14340]: cluster 2024-03-12T02:36:11.437209+0000 mgr.smithi153.asomxr (mgr.14195) 944 : cluster 0 pgmap v786: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:13.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:13 smithi163 bash[22347]: cluster 2024-03-12T02:36:11.437209+0000 mgr.smithi153.asomxr (mgr.14195) 944 : cluster 0 pgmap v786: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:13.826 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:36:13.827 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:36:14.661 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:36:15.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:15 smithi163 bash[22347]: cluster 2024-03-12T02:36:13.437729+0000 mgr.smithi153.asomxr (mgr.14195) 945 : cluster 0 pgmap v787: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:15.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:15 smithi163 bash[22347]: audit 2024-03-12T02:36:13.810252+0000 mgr.smithi153.asomxr (mgr.14195) 946 : audit 0 from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:15.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:15 smithi153 bash[14340]: cluster 2024-03-12T02:36:13.437729+0000 mgr.smithi153.asomxr (mgr.14195) 945 : cluster 0 pgmap v787: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:15.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:15 smithi153 bash[14340]: audit 2024-03-12T02:36:13.810252+0000 mgr.smithi153.asomxr (mgr.14195) 946 : audit 0 from='client.14946 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:15.662 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:36:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:16 smithi163 bash[22347]: audit 2024-03-12T02:36:15.414596+0000 mon.smithi153 (mon.0) 1009 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:36:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:16 smithi153 bash[14340]: audit 2024-03-12T02:36:15.414596+0000 mon.smithi153 (mon.0) 1009 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:36:17.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:17 smithi163 bash[22347]: cluster 2024-03-12T02:36:15.438239+0000 mgr.smithi153.asomxr (mgr.14195) 947 : cluster 0 pgmap v788: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:17.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:17 smithi153 bash[14340]: cluster 2024-03-12T02:36:15.438239+0000 mgr.smithi153.asomxr (mgr.14195) 947 : cluster 0 pgmap v788: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:19.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:19 smithi163 bash[22347]: cluster 2024-03-12T02:36:17.438849+0000 mgr.smithi153.asomxr (mgr.14195) 948 : cluster 0 pgmap v789: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:19 smithi153 bash[14340]: cluster 2024-03-12T02:36:17.438849+0000 mgr.smithi153.asomxr (mgr.14195) 948 : cluster 0 pgmap v789: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:20.416 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:36:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:20 smithi163 bash[22347]: cluster 2024-03-12T02:36:19.439464+0000 mgr.smithi153.asomxr (mgr.14195) 949 : cluster 0 pgmap v790: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:20.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:20 smithi153 bash[14340]: cluster 2024-03-12T02:36:19.439464+0000 mgr.smithi153.asomxr (mgr.14195) 949 : cluster 0 pgmap v790: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:22.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:22 smithi153 bash[14340]: cluster 2024-03-12T02:36:21.439907+0000 mgr.smithi153.asomxr (mgr.14195) 950 : cluster 0 pgmap v791: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:22.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:22 smithi163 bash[22347]: cluster 2024-03-12T02:36:21.439907+0000 mgr.smithi153.asomxr (mgr.14195) 950 : cluster 0 pgmap v791: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:23.564 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:36:23.565 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:36:24.412 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:36:25.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:24 smithi153 bash[14340]: cluster 2024-03-12T02:36:23.440494+0000 mgr.smithi153.asomxr (mgr.14195) 951 : cluster 0 pgmap v792: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:25.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:24 smithi153 bash[14340]: audit 2024-03-12T02:36:23.548911+0000 mgr.smithi153.asomxr (mgr.14195) 952 : audit 0 from='client.14950 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:25.413 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:36:25.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:24 smithi163 bash[22347]: cluster 2024-03-12T02:36:23.440494+0000 mgr.smithi153.asomxr (mgr.14195) 951 : cluster 0 pgmap v792: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:25.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:24 smithi163 bash[22347]: audit 2024-03-12T02:36:23.548911+0000 mgr.smithi153.asomxr (mgr.14195) 952 : audit 0 from='client.14950 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:27.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:26 smithi153 bash[14340]: cluster 2024-03-12T02:36:25.441293+0000 mgr.smithi153.asomxr (mgr.14195) 953 : cluster 0 pgmap v793: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:27.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:27 smithi163 bash[22347]: cluster 2024-03-12T02:36:25.441293+0000 mgr.smithi153.asomxr (mgr.14195) 953 : cluster 0 pgmap v793: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:29.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:29 smithi153 bash[14340]: cluster 2024-03-12T02:36:27.441917+0000 mgr.smithi153.asomxr (mgr.14195) 954 : cluster 0 pgmap v794: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:29 smithi163 bash[22347]: cluster 2024-03-12T02:36:27.441917+0000 mgr.smithi153.asomxr (mgr.14195) 954 : cluster 0 pgmap v794: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:30.166 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:36:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:31 smithi153 bash[14340]: cluster 2024-03-12T02:36:29.442538+0000 mgr.smithi153.asomxr (mgr.14195) 955 : cluster 0 pgmap v795: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:31.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:31 smithi153 bash[14340]: audit 2024-03-12T02:36:30.414852+0000 mon.smithi153 (mon.0) 1010 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:36:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:31 smithi163 bash[22347]: cluster 2024-03-12T02:36:29.442538+0000 mgr.smithi153.asomxr (mgr.14195) 955 : cluster 0 pgmap v795: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:31.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:31 smithi163 bash[22347]: audit 2024-03-12T02:36:30.414852+0000 mon.smithi153 (mon.0) 1010 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:36:33.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:33 smithi153 bash[14340]: cluster 2024-03-12T02:36:31.443131+0000 mgr.smithi153.asomxr (mgr.14195) 956 : cluster 0 pgmap v796: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:33.257 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:36:33.257 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:36:33.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:33 smithi163 bash[22347]: cluster 2024-03-12T02:36:31.443131+0000 mgr.smithi153.asomxr (mgr.14195) 956 : cluster 0 pgmap v796: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:34.046 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:36:35.047 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:36:35.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:35 smithi153 bash[14340]: audit 2024-03-12T02:36:33.240962+0000 mgr.smithi153.asomxr (mgr.14195) 957 : audit 0 from='client.14954 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:35.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:35 smithi153 bash[14340]: cluster 2024-03-12T02:36:33.443533+0000 mgr.smithi153.asomxr (mgr.14195) 958 : cluster 0 pgmap v797: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:35.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:35 smithi163 bash[22347]: audit 2024-03-12T02:36:33.240962+0000 mgr.smithi153.asomxr (mgr.14195) 957 : audit 0 from='client.14954 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:35.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:35 smithi163 bash[22347]: cluster 2024-03-12T02:36:33.443533+0000 mgr.smithi153.asomxr (mgr.14195) 958 : cluster 0 pgmap v797: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:37.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:37 smithi153 bash[14340]: cluster 2024-03-12T02:36:35.444162+0000 mgr.smithi153.asomxr (mgr.14195) 959 : cluster 0 pgmap v798: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:37.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:37 smithi163 bash[22347]: cluster 2024-03-12T02:36:35.444162+0000 mgr.smithi153.asomxr (mgr.14195) 959 : cluster 0 pgmap v798: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:39.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:39 smithi153 bash[14340]: cluster 2024-03-12T02:36:37.444842+0000 mgr.smithi153.asomxr (mgr.14195) 960 : cluster 0 pgmap v799: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:39.392 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:39 smithi163 bash[22347]: cluster 2024-03-12T02:36:37.444842+0000 mgr.smithi153.asomxr (mgr.14195) 960 : cluster 0 pgmap v799: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:39.801 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:36:41.219 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:41 smithi153 bash[14340]: cluster 2024-03-12T02:36:39.445502+0000 mgr.smithi153.asomxr (mgr.14195) 961 : cluster 0 pgmap v800: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:41.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:41 smithi163 bash[22347]: cluster 2024-03-12T02:36:39.445502+0000 mgr.smithi153.asomxr (mgr.14195) 961 : cluster 0 pgmap v800: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:43.214 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:36:43.214 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:36:43.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:43 smithi153 bash[14340]: cluster 2024-03-12T02:36:41.445897+0000 mgr.smithi153.asomxr (mgr.14195) 962 : cluster 0 pgmap v801: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:43.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:43 smithi163 bash[22347]: cluster 2024-03-12T02:36:41.445897+0000 mgr.smithi153.asomxr (mgr.14195) 962 : cluster 0 pgmap v801: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:44.066 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:36:45.067 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:36:45.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:45 smithi153 bash[14340]: audit 2024-03-12T02:36:43.197935+0000 mgr.smithi153.asomxr (mgr.14195) 963 : audit 0 from='client.14958 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:45.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:45 smithi153 bash[14340]: cluster 2024-03-12T02:36:43.446457+0000 mgr.smithi153.asomxr (mgr.14195) 964 : cluster 0 pgmap v802: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:45.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:45 smithi163 bash[22347]: audit 2024-03-12T02:36:43.197935+0000 mgr.smithi153.asomxr (mgr.14195) 963 : audit 0 from='client.14958 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:45.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:45 smithi163 bash[22347]: cluster 2024-03-12T02:36:43.446457+0000 mgr.smithi153.asomxr (mgr.14195) 964 : cluster 0 pgmap v802: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:46 smithi153 bash[14340]: audit 2024-03-12T02:36:45.415188+0000 mon.smithi153 (mon.0) 1011 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:36:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:46 smithi163 bash[22347]: audit 2024-03-12T02:36:45.415188+0000 mon.smithi153 (mon.0) 1011 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:36:47.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:47 smithi153 bash[14340]: cluster 2024-03-12T02:36:45.447111+0000 mgr.smithi153.asomxr (mgr.14195) 965 : cluster 0 pgmap v803: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:47.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:47 smithi163 bash[22347]: cluster 2024-03-12T02:36:45.447111+0000 mgr.smithi153.asomxr (mgr.14195) 965 : cluster 0 pgmap v803: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:49.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:49 smithi153 bash[14340]: cluster 2024-03-12T02:36:47.447703+0000 mgr.smithi153.asomxr (mgr.14195) 966 : cluster 0 pgmap v804: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:49.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:49 smithi163 bash[22347]: cluster 2024-03-12T02:36:47.447703+0000 mgr.smithi153.asomxr (mgr.14195) 966 : cluster 0 pgmap v804: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:49.825 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:36:51.268 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:51 smithi153 bash[14340]: cluster 2024-03-12T02:36:49.448303+0000 mgr.smithi153.asomxr (mgr.14195) 967 : cluster 0 pgmap v805: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:51 smithi163 bash[22347]: cluster 2024-03-12T02:36:49.448303+0000 mgr.smithi153.asomxr (mgr.14195) 967 : cluster 0 pgmap v805: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:53.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:53 smithi153 bash[14340]: cluster 2024-03-12T02:36:51.448959+0000 mgr.smithi153.asomxr (mgr.14195) 968 : cluster 0 pgmap v806: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:53.325 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:36:53.325 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:36:53.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:53 smithi163 bash[22347]: cluster 2024-03-12T02:36:51.448959+0000 mgr.smithi153.asomxr (mgr.14195) 968 : cluster 0 pgmap v806: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:54.166 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:36:55.167 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:36:55.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:55 smithi153 bash[14340]: audit 2024-03-12T02:36:53.307282+0000 mgr.smithi153.asomxr (mgr.14195) 969 : audit 0 from='client.14962 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:55.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:55 smithi153 bash[14340]: cluster 2024-03-12T02:36:53.449624+0000 mgr.smithi153.asomxr (mgr.14195) 970 : cluster 0 pgmap v807: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:55.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:55 smithi163 bash[22347]: audit 2024-03-12T02:36:53.307282+0000 mgr.smithi153.asomxr (mgr.14195) 969 : audit 0 from='client.14962 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:36:55.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:55 smithi163 bash[22347]: cluster 2024-03-12T02:36:53.449624+0000 mgr.smithi153.asomxr (mgr.14195) 970 : cluster 0 pgmap v807: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:57.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:57 smithi163 bash[22347]: cluster 2024-03-12T02:36:55.450210+0000 mgr.smithi153.asomxr (mgr.14195) 971 : cluster 0 pgmap v808: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:57.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:57 smithi153 bash[14340]: cluster 2024-03-12T02:36:55.450210+0000 mgr.smithi153.asomxr (mgr.14195) 971 : cluster 0 pgmap v808: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:36:59 smithi163 bash[22347]: cluster 2024-03-12T02:36:57.450859+0000 mgr.smithi153.asomxr (mgr.14195) 972 : cluster 0 pgmap v809: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:36:59 smithi153 bash[14340]: cluster 2024-03-12T02:36:57.450859+0000 mgr.smithi153.asomxr (mgr.14195) 972 : cluster 0 pgmap v809: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:36:59.917 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:37:01.300 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:01 smithi153 bash[14340]: cluster 2024-03-12T02:36:59.451526+0000 mgr.smithi153.asomxr (mgr.14195) 973 : cluster 0 pgmap v810: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:01.300 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:01 smithi153 bash[14340]: audit 2024-03-12T02:37:00.415661+0000 mon.smithi153 (mon.0) 1012 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:37:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:01 smithi163 bash[22347]: cluster 2024-03-12T02:36:59.451526+0000 mgr.smithi153.asomxr (mgr.14195) 973 : cluster 0 pgmap v810: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:01 smithi163 bash[22347]: audit 2024-03-12T02:37:00.415661+0000 mon.smithi153 (mon.0) 1012 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:37:03.306 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:37:03.307 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:37:03.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:03 smithi153 bash[14340]: cluster 2024-03-12T02:37:01.452176+0000 mgr.smithi153.asomxr (mgr.14195) 974 : cluster 0 pgmap v811: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:03.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:03 smithi163 bash[22347]: cluster 2024-03-12T02:37:01.452176+0000 mgr.smithi153.asomxr (mgr.14195) 974 : cluster 0 pgmap v811: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:04.199 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:37:05.200 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:37:05.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:05 smithi163 bash[22347]: audit 2024-03-12T02:37:03.294959+0000 mgr.smithi153.asomxr (mgr.14195) 975 : audit 0 from='client.14966 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:05.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:05 smithi163 bash[22347]: cluster 2024-03-12T02:37:03.452812+0000 mgr.smithi153.asomxr (mgr.14195) 976 : cluster 0 pgmap v812: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:05.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:05 smithi153 bash[14340]: audit 2024-03-12T02:37:03.294959+0000 mgr.smithi153.asomxr (mgr.14195) 975 : audit 0 from='client.14966 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:05.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:05 smithi153 bash[14340]: cluster 2024-03-12T02:37:03.452812+0000 mgr.smithi153.asomxr (mgr.14195) 976 : cluster 0 pgmap v812: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:07 smithi163 bash[22347]: cluster 2024-03-12T02:37:05.453524+0000 mgr.smithi153.asomxr (mgr.14195) 977 : cluster 0 pgmap v813: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:07.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:07 smithi153 bash[14340]: cluster 2024-03-12T02:37:05.453524+0000 mgr.smithi153.asomxr (mgr.14195) 977 : cluster 0 pgmap v813: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:08.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:08 smithi163 bash[22347]: audit 2024-03-12T02:37:07.535066+0000 mon.smithi153 (mon.0) 1013 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:37:08.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:08 smithi153 bash[14340]: audit 2024-03-12T02:37:07.535066+0000 mon.smithi153 (mon.0) 1013 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:37:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:09 smithi163 bash[22347]: cluster 2024-03-12T02:37:07.454148+0000 mgr.smithi153.asomxr (mgr.14195) 978 : cluster 0 pgmap v814: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:09.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:09 smithi163 bash[22347]: audit 2024-03-12T02:37:08.117541+0000 mon.smithi153 (mon.0) 1014 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:37:09.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:09 smithi163 bash[22347]: audit 2024-03-12T02:37:08.119054+0000 mon.smithi153 (mon.0) 1015 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:37:09.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:09 smithi163 bash[22347]: cluster 2024-03-12T02:37:08.122080+0000 mgr.smithi153.asomxr (mgr.14195) 979 : cluster 0 pgmap v815: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:09.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:09 smithi163 bash[22347]: cluster 2024-03-12T02:37:08.122499+0000 mgr.smithi153.asomxr (mgr.14195) 980 : cluster 0 pgmap v816: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:09.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:09 smithi163 bash[22347]: audit 2024-03-12T02:37:08.129329+0000 mon.smithi153 (mon.0) 1016 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:37:09.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:09 smithi163 bash[22347]: audit 2024-03-12T02:37:08.134338+0000 mon.smithi153 (mon.0) 1017 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:37:09.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:09 smithi153 bash[14340]: cluster 2024-03-12T02:37:07.454148+0000 mgr.smithi153.asomxr (mgr.14195) 978 : cluster 0 pgmap v814: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:09.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:09 smithi153 bash[14340]: audit 2024-03-12T02:37:08.117541+0000 mon.smithi153 (mon.0) 1014 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:37:09.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:09 smithi153 bash[14340]: audit 2024-03-12T02:37:08.119054+0000 mon.smithi153 (mon.0) 1015 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:37:09.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:09 smithi153 bash[14340]: cluster 2024-03-12T02:37:08.122080+0000 mgr.smithi153.asomxr (mgr.14195) 979 : cluster 0 pgmap v815: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:09.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:09 smithi153 bash[14340]: cluster 2024-03-12T02:37:08.122499+0000 mgr.smithi153.asomxr (mgr.14195) 980 : cluster 0 pgmap v816: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:09.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:09 smithi153 bash[14340]: audit 2024-03-12T02:37:08.129329+0000 mon.smithi153 (mon.0) 1016 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:37:09.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:09 smithi153 bash[14340]: audit 2024-03-12T02:37:08.134338+0000 mon.smithi153 (mon.0) 1017 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:37:09.954 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:37:10.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:10 smithi153 bash[14340]: cluster 2024-03-12T02:37:10.122933+0000 mgr.smithi153.asomxr (mgr.14195) 981 : cluster 0 pgmap v817: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:10.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:10 smithi163 bash[22347]: cluster 2024-03-12T02:37:10.122933+0000 mgr.smithi153.asomxr (mgr.14195) 981 : cluster 0 pgmap v817: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:12.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:12 smithi153 bash[14340]: cluster 2024-03-12T02:37:12.123351+0000 mgr.smithi153.asomxr (mgr.14195) 982 : cluster 0 pgmap v818: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:12 smithi163 bash[22347]: cluster 2024-03-12T02:37:12.123351+0000 mgr.smithi153.asomxr (mgr.14195) 982 : cluster 0 pgmap v818: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:13.499 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:37:13.500 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:37:14.298 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:37:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:14 smithi163 bash[22347]: audit 2024-03-12T02:37:13.481527+0000 mgr.smithi153.asomxr (mgr.14195) 983 : audit 0 from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:14 smithi163 bash[22347]: cluster 2024-03-12T02:37:14.123918+0000 mgr.smithi153.asomxr (mgr.14195) 984 : cluster 0 pgmap v819: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:14.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:14 smithi153 bash[14340]: audit 2024-03-12T02:37:13.481527+0000 mgr.smithi153.asomxr (mgr.14195) 983 : audit 0 from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:14.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:14 smithi153 bash[14340]: cluster 2024-03-12T02:37:14.123918+0000 mgr.smithi153.asomxr (mgr.14195) 984 : cluster 0 pgmap v819: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:15.299 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:37:15.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:15 smithi153 bash[14340]: audit 2024-03-12T02:37:15.415730+0000 mon.smithi153 (mon.0) 1018 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:37:15.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:15 smithi163 bash[22347]: audit 2024-03-12T02:37:15.415730+0000 mon.smithi153 (mon.0) 1018 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:37:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:16 smithi163 bash[22347]: cluster 2024-03-12T02:37:16.124550+0000 mgr.smithi153.asomxr (mgr.14195) 985 : cluster 0 pgmap v820: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:16.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:16 smithi153 bash[14340]: cluster 2024-03-12T02:37:16.124550+0000 mgr.smithi153.asomxr (mgr.14195) 985 : cluster 0 pgmap v820: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:19.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:19 smithi153 bash[14340]: cluster 2024-03-12T02:37:18.125250+0000 mgr.smithi153.asomxr (mgr.14195) 986 : cluster 0 pgmap v821: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:19 smithi163 bash[22347]: cluster 2024-03-12T02:37:18.125250+0000 mgr.smithi153.asomxr (mgr.14195) 986 : cluster 0 pgmap v821: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:20.057 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:37:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:20 smithi163 bash[22347]: cluster 2024-03-12T02:37:20.125861+0000 mgr.smithi153.asomxr (mgr.14195) 987 : cluster 0 pgmap v822: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:20.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:20 smithi153 bash[14340]: cluster 2024-03-12T02:37:20.125861+0000 mgr.smithi153.asomxr (mgr.14195) 987 : cluster 0 pgmap v822: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:22.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:22 smithi163 bash[22347]: cluster 2024-03-12T02:37:22.126261+0000 mgr.smithi153.asomxr (mgr.14195) 988 : cluster 0 pgmap v823: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:22.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:22 smithi153 bash[14340]: cluster 2024-03-12T02:37:22.126261+0000 mgr.smithi153.asomxr (mgr.14195) 988 : cluster 0 pgmap v823: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:23.680 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:37:23.680 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:37:24.453 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:37:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:24 smithi163 bash[22347]: audit 2024-03-12T02:37:23.662104+0000 mgr.smithi153.asomxr (mgr.14195) 989 : audit 0 from='client.14974 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:24 smithi163 bash[22347]: cluster 2024-03-12T02:37:24.126849+0000 mgr.smithi153.asomxr (mgr.14195) 990 : cluster 0 pgmap v824: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:24.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:24 smithi153 bash[14340]: audit 2024-03-12T02:37:23.662104+0000 mgr.smithi153.asomxr (mgr.14195) 989 : audit 0 from='client.14974 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:24.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:24 smithi153 bash[14340]: cluster 2024-03-12T02:37:24.126849+0000 mgr.smithi153.asomxr (mgr.14195) 990 : cluster 0 pgmap v824: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:25.454 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:37:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:26 smithi163 bash[22347]: cluster 2024-03-12T02:37:26.127510+0000 mgr.smithi153.asomxr (mgr.14195) 991 : cluster 0 pgmap v825: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:26 smithi153 bash[14340]: cluster 2024-03-12T02:37:26.127510+0000 mgr.smithi153.asomxr (mgr.14195) 991 : cluster 0 pgmap v825: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:29.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:29 smithi153 bash[14340]: cluster 2024-03-12T02:37:28.128106+0000 mgr.smithi153.asomxr (mgr.14195) 992 : cluster 0 pgmap v826: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:29 smithi163 bash[22347]: cluster 2024-03-12T02:37:28.128106+0000 mgr.smithi153.asomxr (mgr.14195) 992 : cluster 0 pgmap v826: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:30.205 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:37:30.463 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:30 smithi163 bash[22347]: cluster 2024-03-12T02:37:30.128704+0000 mgr.smithi153.asomxr (mgr.14195) 993 : cluster 0 pgmap v827: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:30.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:30 smithi153 bash[14340]: cluster 2024-03-12T02:37:30.128704+0000 mgr.smithi153.asomxr (mgr.14195) 993 : cluster 0 pgmap v827: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:31 smithi163 bash[22347]: audit 2024-03-12T02:37:30.416419+0000 mon.smithi153 (mon.0) 1019 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:37:31.520 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:31 smithi153 bash[14340]: audit 2024-03-12T02:37:30.416419+0000 mon.smithi153 (mon.0) 1019 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:37:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:32 smithi163 bash[22347]: cluster 2024-03-12T02:37:32.129404+0000 mgr.smithi153.asomxr (mgr.14195) 994 : cluster 0 pgmap v828: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:32.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:32 smithi153 bash[14340]: cluster 2024-03-12T02:37:32.129404+0000 mgr.smithi153.asomxr (mgr.14195) 994 : cluster 0 pgmap v828: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:33.519 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:37:33.519 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:37:34.316 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:37:34.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:34 smithi163 bash[22347]: audit 2024-03-12T02:37:33.502197+0000 mgr.smithi153.asomxr (mgr.14195) 995 : audit 0 from='client.14978 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:34.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:34 smithi163 bash[22347]: cluster 2024-03-12T02:37:34.130026+0000 mgr.smithi153.asomxr (mgr.14195) 996 : cluster 0 pgmap v829: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:34.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:34 smithi153 bash[14340]: audit 2024-03-12T02:37:33.502197+0000 mgr.smithi153.asomxr (mgr.14195) 995 : audit 0 from='client.14978 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:34.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:34 smithi153 bash[14340]: cluster 2024-03-12T02:37:34.130026+0000 mgr.smithi153.asomxr (mgr.14195) 996 : cluster 0 pgmap v829: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:35.317 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:37:36.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:36 smithi163 bash[22347]: cluster 2024-03-12T02:37:36.130601+0000 mgr.smithi153.asomxr (mgr.14195) 997 : cluster 0 pgmap v830: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:36.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:36 smithi153 bash[14340]: cluster 2024-03-12T02:37:36.130601+0000 mgr.smithi153.asomxr (mgr.14195) 997 : cluster 0 pgmap v830: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:39.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:39 smithi153 bash[14340]: cluster 2024-03-12T02:37:38.131200+0000 mgr.smithi153.asomxr (mgr.14195) 998 : cluster 0 pgmap v831: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:39.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:39 smithi163 bash[22347]: cluster 2024-03-12T02:37:38.131200+0000 mgr.smithi153.asomxr (mgr.14195) 998 : cluster 0 pgmap v831: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:40.071 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:37:40.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:40 smithi163 bash[22347]: cluster 2024-03-12T02:37:40.131634+0000 mgr.smithi153.asomxr (mgr.14195) 999 : cluster 0 pgmap v832: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:40.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:40 smithi153 bash[14340]: cluster 2024-03-12T02:37:40.131634+0000 mgr.smithi153.asomxr (mgr.14195) 999 : cluster 0 pgmap v832: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:42.481 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:42 smithi153 bash[14340]: cluster 2024-03-12T02:37:42.132297+0000 mgr.smithi153.asomxr (mgr.14195) 1000 : cluster 0 pgmap v833: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:42.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:42 smithi163 bash[22347]: cluster 2024-03-12T02:37:42.132297+0000 mgr.smithi153.asomxr (mgr.14195) 1000 : cluster 0 pgmap v833: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:43.491 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:37:43.491 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:37:44.368 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:37:44.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:44 smithi163 bash[22347]: audit 2024-03-12T02:37:43.477480+0000 mgr.smithi153.asomxr (mgr.14195) 1001 : audit 0 from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:44.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:44 smithi163 bash[22347]: cluster 2024-03-12T02:37:44.132944+0000 mgr.smithi153.asomxr (mgr.14195) 1002 : cluster 0 pgmap v834: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:44.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:44 smithi153 bash[14340]: audit 2024-03-12T02:37:43.477480+0000 mgr.smithi153.asomxr (mgr.14195) 1001 : audit 0 from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:44.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:44 smithi153 bash[14340]: cluster 2024-03-12T02:37:44.132944+0000 mgr.smithi153.asomxr (mgr.14195) 1002 : cluster 0 pgmap v834: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:45.369 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:37:45.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:45 smithi163 bash[22347]: audit 2024-03-12T02:37:45.416405+0000 mon.smithi153 (mon.0) 1020 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:37:45.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:45 smithi153 bash[14340]: audit 2024-03-12T02:37:45.416405+0000 mon.smithi153 (mon.0) 1020 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:37:46.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:46 smithi153 bash[14340]: cluster 2024-03-12T02:37:46.133586+0000 mgr.smithi153.asomxr (mgr.14195) 1003 : cluster 0 pgmap v835: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:46.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:46 smithi163 bash[22347]: cluster 2024-03-12T02:37:46.133586+0000 mgr.smithi153.asomxr (mgr.14195) 1003 : cluster 0 pgmap v835: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:49.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:49 smithi153 bash[14340]: cluster 2024-03-12T02:37:48.134127+0000 mgr.smithi153.asomxr (mgr.14195) 1004 : cluster 0 pgmap v836: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:49.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:49 smithi163 bash[22347]: cluster 2024-03-12T02:37:48.134127+0000 mgr.smithi153.asomxr (mgr.14195) 1004 : cluster 0 pgmap v836: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:50.119 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:37:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:50 smithi163 bash[22347]: cluster 2024-03-12T02:37:50.134852+0000 mgr.smithi153.asomxr (mgr.14195) 1005 : cluster 0 pgmap v837: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:50.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:50 smithi153 bash[14340]: cluster 2024-03-12T02:37:50.134852+0000 mgr.smithi153.asomxr (mgr.14195) 1005 : cluster 0 pgmap v837: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:52 smithi163 bash[22347]: cluster 2024-03-12T02:37:52.135487+0000 mgr.smithi153.asomxr (mgr.14195) 1006 : cluster 0 pgmap v838: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:52.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:52 smithi153 bash[14340]: cluster 2024-03-12T02:37:52.135487+0000 mgr.smithi153.asomxr (mgr.14195) 1006 : cluster 0 pgmap v838: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:53.694 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:37:53.694 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:37:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:54 smithi163 bash[22347]: audit 2024-03-12T02:37:53.678632+0000 mgr.smithi153.asomxr (mgr.14195) 1007 : audit 0 from='client.14986 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:54.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:54 smithi163 bash[22347]: cluster 2024-03-12T02:37:54.136063+0000 mgr.smithi153.asomxr (mgr.14195) 1008 : cluster 0 pgmap v839: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:54.539 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:54 smithi153 bash[14340]: audit 2024-03-12T02:37:53.678632+0000 mgr.smithi153.asomxr (mgr.14195) 1007 : audit 0 from='client.14986 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:37:54.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:54 smithi153 bash[14340]: cluster 2024-03-12T02:37:54.136063+0000 mgr.smithi153.asomxr (mgr.14195) 1008 : cluster 0 pgmap v839: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:54.541 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:37:55.542 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:37:56.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:56 smithi163 bash[22347]: cluster 2024-03-12T02:37:56.136650+0000 mgr.smithi153.asomxr (mgr.14195) 1009 : cluster 0 pgmap v840: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:56.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:56 smithi153 bash[14340]: cluster 2024-03-12T02:37:56.136650+0000 mgr.smithi153.asomxr (mgr.14195) 1009 : cluster 0 pgmap v840: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:37:59 smithi153 bash[14340]: cluster 2024-03-12T02:37:58.137298+0000 mgr.smithi153.asomxr (mgr.14195) 1010 : cluster 0 pgmap v841: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:37:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:37:59 smithi163 bash[22347]: cluster 2024-03-12T02:37:58.137298+0000 mgr.smithi153.asomxr (mgr.14195) 1010 : cluster 0 pgmap v841: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:00.291 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:38:00.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:00 smithi163 bash[22347]: cluster 2024-03-12T02:38:00.137884+0000 mgr.smithi153.asomxr (mgr.14195) 1011 : cluster 0 pgmap v842: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:00.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:00 smithi153 bash[14340]: cluster 2024-03-12T02:38:00.137884+0000 mgr.smithi153.asomxr (mgr.14195) 1011 : cluster 0 pgmap v842: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:01 smithi163 bash[22347]: audit 2024-03-12T02:38:00.417081+0000 mon.smithi153 (mon.0) 1021 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:38:01.512 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:01 smithi153 bash[14340]: audit 2024-03-12T02:38:00.417081+0000 mon.smithi153 (mon.0) 1021 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:38:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:02 smithi163 bash[22347]: cluster 2024-03-12T02:38:02.138426+0000 mgr.smithi153.asomxr (mgr.14195) 1012 : cluster 0 pgmap v843: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:02.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:02 smithi153 bash[14340]: cluster 2024-03-12T02:38:02.138426+0000 mgr.smithi153.asomxr (mgr.14195) 1012 : cluster 0 pgmap v843: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:03.768 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:38:03.769 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:38:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:04 smithi163 bash[22347]: audit 2024-03-12T02:38:03.750758+0000 mgr.smithi153.asomxr (mgr.14195) 1013 : audit 0 from='client.14990 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:04 smithi163 bash[22347]: cluster 2024-03-12T02:38:04.139059+0000 mgr.smithi153.asomxr (mgr.14195) 1014 : cluster 0 pgmap v844: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:04.568 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:04 smithi153 bash[14340]: audit 2024-03-12T02:38:03.750758+0000 mgr.smithi153.asomxr (mgr.14195) 1013 : audit 0 from='client.14990 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:04.568 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:04 smithi153 bash[14340]: cluster 2024-03-12T02:38:04.139059+0000 mgr.smithi153.asomxr (mgr.14195) 1014 : cluster 0 pgmap v844: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:04.569 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:38:05.570 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:38:06.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:06 smithi163 bash[22347]: cluster 2024-03-12T02:38:06.139736+0000 mgr.smithi153.asomxr (mgr.14195) 1015 : cluster 0 pgmap v845: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:06 smithi153 bash[14340]: cluster 2024-03-12T02:38:06.139736+0000 mgr.smithi153.asomxr (mgr.14195) 1015 : cluster 0 pgmap v845: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:09.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:09 smithi153 bash[14340]: cluster 2024-03-12T02:38:08.140332+0000 mgr.smithi153.asomxr (mgr.14195) 1016 : cluster 0 pgmap v846: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:09.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:09 smithi153 bash[14340]: audit 2024-03-12T02:38:08.232888+0000 mon.smithi153 (mon.0) 1022 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:38:09.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:09 smithi153 bash[14340]: audit 2024-03-12T02:38:08.813484+0000 mon.smithi153 (mon.0) 1023 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:38:09.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:09 smithi153 bash[14340]: audit 2024-03-12T02:38:08.815414+0000 mon.smithi153 (mon.0) 1024 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:38:09.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:09 smithi153 bash[14340]: audit 2024-03-12T02:38:08.825705+0000 mon.smithi153 (mon.0) 1025 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:38:09.325 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:09 smithi153 bash[14340]: audit 2024-03-12T02:38:08.830418+0000 mon.smithi153 (mon.0) 1026 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:38:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:09 smithi163 bash[22347]: cluster 2024-03-12T02:38:08.140332+0000 mgr.smithi153.asomxr (mgr.14195) 1016 : cluster 0 pgmap v846: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:09.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:09 smithi163 bash[22347]: audit 2024-03-12T02:38:08.232888+0000 mon.smithi153 (mon.0) 1022 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:38:09.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:09 smithi163 bash[22347]: audit 2024-03-12T02:38:08.813484+0000 mon.smithi153 (mon.0) 1023 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:38:09.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:09 smithi163 bash[22347]: audit 2024-03-12T02:38:08.815414+0000 mon.smithi153 (mon.0) 1024 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:38:09.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:09 smithi163 bash[22347]: audit 2024-03-12T02:38:08.825705+0000 mon.smithi153 (mon.0) 1025 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:38:09.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:09 smithi163 bash[22347]: audit 2024-03-12T02:38:08.830418+0000 mon.smithi153 (mon.0) 1026 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:38:10.323 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:38:10.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:10 smithi153 bash[14340]: cluster 2024-03-12T02:38:08.818353+0000 mgr.smithi153.asomxr (mgr.14195) 1017 : cluster 0 pgmap v847: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:10.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:10 smithi153 bash[14340]: cluster 2024-03-12T02:38:08.818729+0000 mgr.smithi153.asomxr (mgr.14195) 1018 : cluster 0 pgmap v848: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:10.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:10 smithi163 bash[22347]: cluster 2024-03-12T02:38:08.818353+0000 mgr.smithi153.asomxr (mgr.14195) 1017 : cluster 0 pgmap v847: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:10.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:10 smithi163 bash[22347]: cluster 2024-03-12T02:38:08.818729+0000 mgr.smithi153.asomxr (mgr.14195) 1018 : cluster 0 pgmap v848: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:12.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:12 smithi153 bash[14340]: cluster 2024-03-12T02:38:10.819285+0000 mgr.smithi153.asomxr (mgr.14195) 1019 : cluster 0 pgmap v849: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:12 smithi163 bash[22347]: cluster 2024-03-12T02:38:10.819285+0000 mgr.smithi153.asomxr (mgr.14195) 1019 : cluster 0 pgmap v849: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:13.658 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:38:13.658 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:38:14.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:14 smithi153 bash[14340]: cluster 2024-03-12T02:38:12.819671+0000 mgr.smithi153.asomxr (mgr.14195) 1020 : cluster 0 pgmap v850: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:14.463 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:38:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:14 smithi163 bash[22347]: cluster 2024-03-12T02:38:12.819671+0000 mgr.smithi153.asomxr (mgr.14195) 1020 : cluster 0 pgmap v850: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:15.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:15 smithi153 bash[14340]: audit 2024-03-12T02:38:13.643065+0000 mgr.smithi153.asomxr (mgr.14195) 1021 : audit 0 from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:15.464 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:38:15.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:15 smithi163 bash[22347]: audit 2024-03-12T02:38:13.643065+0000 mgr.smithi153.asomxr (mgr.14195) 1021 : audit 0 from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:16 smithi153 bash[14340]: cluster 2024-03-12T02:38:14.820209+0000 mgr.smithi153.asomxr (mgr.14195) 1022 : cluster 0 pgmap v851: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:16.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:16 smithi153 bash[14340]: audit 2024-03-12T02:38:15.417734+0000 mon.smithi153 (mon.0) 1027 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:38:16.404 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:16 smithi163 bash[22347]: cluster 2024-03-12T02:38:14.820209+0000 mgr.smithi153.asomxr (mgr.14195) 1022 : cluster 0 pgmap v851: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:16.404 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:16 smithi163 bash[22347]: audit 2024-03-12T02:38:15.417734+0000 mon.smithi153 (mon.0) 1027 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:38:18.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:18 smithi153 bash[14340]: cluster 2024-03-12T02:38:16.820827+0000 mgr.smithi153.asomxr (mgr.14195) 1023 : cluster 0 pgmap v852: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:18.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:18 smithi163 bash[22347]: cluster 2024-03-12T02:38:16.820827+0000 mgr.smithi153.asomxr (mgr.14195) 1023 : cluster 0 pgmap v852: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:20.211 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:38:20.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:20 smithi153 bash[14340]: cluster 2024-03-12T02:38:18.821433+0000 mgr.smithi153.asomxr (mgr.14195) 1024 : cluster 0 pgmap v853: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:20 smithi163 bash[22347]: cluster 2024-03-12T02:38:18.821433+0000 mgr.smithi153.asomxr (mgr.14195) 1024 : cluster 0 pgmap v853: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:22.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:22 smithi153 bash[14340]: cluster 2024-03-12T02:38:20.822049+0000 mgr.smithi153.asomxr (mgr.14195) 1025 : cluster 0 pgmap v854: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:22.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:22 smithi163 bash[22347]: cluster 2024-03-12T02:38:20.822049+0000 mgr.smithi153.asomxr (mgr.14195) 1025 : cluster 0 pgmap v854: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:23.440 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:38:23.441 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:38:24.317 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:38:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:24 smithi163 bash[22347]: cluster 2024-03-12T02:38:22.822672+0000 mgr.smithi153.asomxr (mgr.14195) 1026 : cluster 0 pgmap v855: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:24.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:24 smithi153 bash[14340]: cluster 2024-03-12T02:38:22.822672+0000 mgr.smithi153.asomxr (mgr.14195) 1026 : cluster 0 pgmap v855: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:25.318 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:38:25.331 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:25 smithi153 bash[14340]: audit 2024-03-12T02:38:23.427723+0000 mgr.smithi153.asomxr (mgr.14195) 1027 : audit 0 from='client.14998 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:25.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:25 smithi163 bash[22347]: audit 2024-03-12T02:38:23.427723+0000 mgr.smithi153.asomxr (mgr.14195) 1027 : audit 0 from='client.14998 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:26 smithi163 bash[22347]: cluster 2024-03-12T02:38:24.823318+0000 mgr.smithi153.asomxr (mgr.14195) 1028 : cluster 0 pgmap v856: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:26 smithi153 bash[14340]: cluster 2024-03-12T02:38:24.823318+0000 mgr.smithi153.asomxr (mgr.14195) 1028 : cluster 0 pgmap v856: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:28 smithi163 bash[22347]: cluster 2024-03-12T02:38:26.823922+0000 mgr.smithi153.asomxr (mgr.14195) 1029 : cluster 0 pgmap v857: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:28.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:28 smithi153 bash[14340]: cluster 2024-03-12T02:38:26.823922+0000 mgr.smithi153.asomxr (mgr.14195) 1029 : cluster 0 pgmap v857: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:30.068 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:38:30.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:30 smithi163 bash[22347]: cluster 2024-03-12T02:38:28.824458+0000 mgr.smithi153.asomxr (mgr.14195) 1030 : cluster 0 pgmap v858: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:30.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:30 smithi153 bash[14340]: cluster 2024-03-12T02:38:28.824458+0000 mgr.smithi153.asomxr (mgr.14195) 1030 : cluster 0 pgmap v858: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:31.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:31 smithi153 bash[14340]: audit 2024-03-12T02:38:30.418056+0000 mon.smithi153 (mon.0) 1028 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:38:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:31 smithi163 bash[22347]: audit 2024-03-12T02:38:30.418056+0000 mon.smithi153 (mon.0) 1028 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:38:32.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:32 smithi153 bash[14340]: cluster 2024-03-12T02:38:30.825098+0000 mgr.smithi153.asomxr (mgr.14195) 1031 : cluster 0 pgmap v859: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:32 smithi163 bash[22347]: cluster 2024-03-12T02:38:30.825098+0000 mgr.smithi153.asomxr (mgr.14195) 1031 : cluster 0 pgmap v859: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:33.220 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:38:33.221 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:38:34.017 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:38:34.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:34 smithi153 bash[14340]: cluster 2024-03-12T02:38:32.825629+0000 mgr.smithi153.asomxr (mgr.14195) 1032 : cluster 0 pgmap v860: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:34.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:34 smithi163 bash[22347]: cluster 2024-03-12T02:38:32.825629+0000 mgr.smithi153.asomxr (mgr.14195) 1032 : cluster 0 pgmap v860: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:35.018 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:38:35.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:35 smithi153 bash[14340]: audit 2024-03-12T02:38:33.204121+0000 mgr.smithi153.asomxr (mgr.14195) 1033 : audit 0 from='client.15002 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:35.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:35 smithi163 bash[22347]: audit 2024-03-12T02:38:33.204121+0000 mgr.smithi153.asomxr (mgr.14195) 1033 : audit 0 from='client.15002 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:36.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:36 smithi163 bash[22347]: cluster 2024-03-12T02:38:34.826201+0000 mgr.smithi153.asomxr (mgr.14195) 1034 : cluster 0 pgmap v861: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:36.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:36 smithi153 bash[14340]: cluster 2024-03-12T02:38:34.826201+0000 mgr.smithi153.asomxr (mgr.14195) 1034 : cluster 0 pgmap v861: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:38.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:38 smithi163 bash[22347]: cluster 2024-03-12T02:38:36.826755+0000 mgr.smithi153.asomxr (mgr.14195) 1035 : cluster 0 pgmap v862: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:38.542 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:38 smithi153 bash[14340]: cluster 2024-03-12T02:38:36.826755+0000 mgr.smithi153.asomxr (mgr.14195) 1035 : cluster 0 pgmap v862: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:39.776 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:38:40.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:40 smithi163 bash[22347]: cluster 2024-03-12T02:38:38.827343+0000 mgr.smithi153.asomxr (mgr.14195) 1036 : cluster 0 pgmap v863: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:40.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:40 smithi153 bash[14340]: cluster 2024-03-12T02:38:38.827343+0000 mgr.smithi153.asomxr (mgr.14195) 1036 : cluster 0 pgmap v863: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:42.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:42 smithi163 bash[22347]: cluster 2024-03-12T02:38:40.827972+0000 mgr.smithi153.asomxr (mgr.14195) 1037 : cluster 0 pgmap v864: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:42.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:42 smithi153 bash[14340]: cluster 2024-03-12T02:38:40.827972+0000 mgr.smithi153.asomxr (mgr.14195) 1037 : cluster 0 pgmap v864: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:43.208 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:38:43.208 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:38:44.006 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:38:44.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:44 smithi153 bash[14340]: cluster 2024-03-12T02:38:42.828680+0000 mgr.smithi153.asomxr (mgr.14195) 1038 : cluster 0 pgmap v865: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:44.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:44 smithi163 bash[22347]: cluster 2024-03-12T02:38:42.828680+0000 mgr.smithi153.asomxr (mgr.14195) 1038 : cluster 0 pgmap v865: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:45.008 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:38:45.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:45 smithi153 bash[14340]: audit 2024-03-12T02:38:43.191149+0000 mgr.smithi153.asomxr (mgr.14195) 1039 : audit 0 from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:45.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:45 smithi163 bash[22347]: audit 2024-03-12T02:38:43.191149+0000 mgr.smithi153.asomxr (mgr.14195) 1039 : audit 0 from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:46 smithi163 bash[22347]: cluster 2024-03-12T02:38:44.829474+0000 mgr.smithi153.asomxr (mgr.14195) 1040 : cluster 0 pgmap v866: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:46 smithi163 bash[22347]: audit 2024-03-12T02:38:45.418380+0000 mon.smithi153 (mon.0) 1029 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:38:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:46 smithi153 bash[14340]: cluster 2024-03-12T02:38:44.829474+0000 mgr.smithi153.asomxr (mgr.14195) 1040 : cluster 0 pgmap v866: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:46 smithi153 bash[14340]: audit 2024-03-12T02:38:45.418380+0000 mon.smithi153 (mon.0) 1029 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:38:48.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:48 smithi163 bash[22347]: cluster 2024-03-12T02:38:46.830027+0000 mgr.smithi153.asomxr (mgr.14195) 1041 : cluster 0 pgmap v867: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:48.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:48 smithi153 bash[14340]: cluster 2024-03-12T02:38:46.830027+0000 mgr.smithi153.asomxr (mgr.14195) 1041 : cluster 0 pgmap v867: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:49.758 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:38:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:50 smithi163 bash[22347]: cluster 2024-03-12T02:38:48.830586+0000 mgr.smithi153.asomxr (mgr.14195) 1042 : cluster 0 pgmap v868: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:50.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:50 smithi153 bash[14340]: cluster 2024-03-12T02:38:48.830586+0000 mgr.smithi153.asomxr (mgr.14195) 1042 : cluster 0 pgmap v868: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:52 smithi163 bash[22347]: cluster 2024-03-12T02:38:50.831202+0000 mgr.smithi153.asomxr (mgr.14195) 1043 : cluster 0 pgmap v869: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:52.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:52 smithi153 bash[14340]: cluster 2024-03-12T02:38:50.831202+0000 mgr.smithi153.asomxr (mgr.14195) 1043 : cluster 0 pgmap v869: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:53.242 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:38:53.242 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:38:54.012 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:38:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:54 smithi163 bash[22347]: cluster 2024-03-12T02:38:52.831635+0000 mgr.smithi153.asomxr (mgr.14195) 1044 : cluster 0 pgmap v870: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:54.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:54 smithi153 bash[14340]: cluster 2024-03-12T02:38:52.831635+0000 mgr.smithi153.asomxr (mgr.14195) 1044 : cluster 0 pgmap v870: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:55.014 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:38:55.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:55 smithi153 bash[14340]: audit 2024-03-12T02:38:53.226851+0000 mgr.smithi153.asomxr (mgr.14195) 1045 : audit 0 from='client.15010 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:55.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:55 smithi163 bash[22347]: audit 2024-03-12T02:38:53.226851+0000 mgr.smithi153.asomxr (mgr.14195) 1045 : audit 0 from='client.15010 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:38:56.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:56 smithi163 bash[22347]: cluster 2024-03-12T02:38:54.832181+0000 mgr.smithi153.asomxr (mgr.14195) 1046 : cluster 0 pgmap v871: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:56.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:56 smithi153 bash[14340]: cluster 2024-03-12T02:38:54.832181+0000 mgr.smithi153.asomxr (mgr.14195) 1046 : cluster 0 pgmap v871: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:57.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:38:57 smithi163 bash[22347]: cluster 2024-03-12T02:38:56.832848+0000 mgr.smithi153.asomxr (mgr.14195) 1047 : cluster 0 pgmap v872: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:57.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:38:57 smithi153 bash[14340]: cluster 2024-03-12T02:38:56.832848+0000 mgr.smithi153.asomxr (mgr.14195) 1047 : cluster 0 pgmap v872: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:38:59.769 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:39:00.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:00 smithi153 bash[14340]: cluster 2024-03-12T02:38:58.833488+0000 mgr.smithi153.asomxr (mgr.14195) 1048 : cluster 0 pgmap v873: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:00.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:00 smithi163 bash[22347]: cluster 2024-03-12T02:38:58.833488+0000 mgr.smithi153.asomxr (mgr.14195) 1048 : cluster 0 pgmap v873: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:01.188 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:01 smithi153 bash[14340]: audit 2024-03-12T02:39:00.418712+0000 mon.smithi153 (mon.0) 1030 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:39:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:01 smithi163 bash[22347]: audit 2024-03-12T02:39:00.418712+0000 mon.smithi153 (mon.0) 1030 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:39:02.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:02 smithi153 bash[14340]: cluster 2024-03-12T02:39:00.834072+0000 mgr.smithi153.asomxr (mgr.14195) 1049 : cluster 0 pgmap v874: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:02.353 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:02 smithi163 bash[22347]: cluster 2024-03-12T02:39:00.834072+0000 mgr.smithi153.asomxr (mgr.14195) 1049 : cluster 0 pgmap v874: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:03.158 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:39:03.158 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:39:03.900 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:39:04.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:04 smithi153 bash[14340]: cluster 2024-03-12T02:39:02.834691+0000 mgr.smithi153.asomxr (mgr.14195) 1050 : cluster 0 pgmap v875: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:04.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:04 smithi153 bash[14340]: audit 2024-03-12T02:39:03.141768+0000 mgr.smithi153.asomxr (mgr.14195) 1051 : audit 0 from='client.15014 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:04 smithi163 bash[22347]: cluster 2024-03-12T02:39:02.834691+0000 mgr.smithi153.asomxr (mgr.14195) 1050 : cluster 0 pgmap v875: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:04 smithi163 bash[22347]: audit 2024-03-12T02:39:03.141768+0000 mgr.smithi153.asomxr (mgr.14195) 1051 : audit 0 from='client.15014 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:04.901 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:39:06.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:06 smithi153 bash[14340]: cluster 2024-03-12T02:39:04.835331+0000 mgr.smithi153.asomxr (mgr.14195) 1052 : cluster 0 pgmap v876: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:06.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:06 smithi163 bash[22347]: cluster 2024-03-12T02:39:04.835331+0000 mgr.smithi153.asomxr (mgr.14195) 1052 : cluster 0 pgmap v876: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:08.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:08 smithi153 bash[14340]: cluster 2024-03-12T02:39:06.835975+0000 mgr.smithi153.asomxr (mgr.14195) 1053 : cluster 0 pgmap v877: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:08.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:08 smithi163 bash[22347]: cluster 2024-03-12T02:39:06.835975+0000 mgr.smithi153.asomxr (mgr.14195) 1053 : cluster 0 pgmap v877: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:09 smithi163 bash[22347]: audit 2024-03-12T02:39:08.924988+0000 mon.smithi153 (mon.0) 1031 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:39:09.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:09 smithi153 bash[14340]: audit 2024-03-12T02:39:08.924988+0000 mon.smithi153 (mon.0) 1031 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:39:09.645 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:39:10.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:10 smithi163 bash[22347]: cluster 2024-03-12T02:39:08.836609+0000 mgr.smithi153.asomxr (mgr.14195) 1054 : cluster 0 pgmap v878: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:10.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:10 smithi163 bash[22347]: audit 2024-03-12T02:39:09.509755+0000 mon.smithi153 (mon.0) 1032 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:39:10.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:10 smithi163 bash[22347]: audit 2024-03-12T02:39:09.511320+0000 mon.smithi153 (mon.0) 1033 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:39:10.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:10 smithi163 bash[22347]: audit 2024-03-12T02:39:09.520126+0000 mon.smithi153 (mon.0) 1034 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:39:10.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:10 smithi163 bash[22347]: audit 2024-03-12T02:39:09.524417+0000 mon.smithi153 (mon.0) 1035 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:39:10.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:10 smithi153 bash[14340]: cluster 2024-03-12T02:39:08.836609+0000 mgr.smithi153.asomxr (mgr.14195) 1054 : cluster 0 pgmap v878: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:10.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:10 smithi153 bash[14340]: audit 2024-03-12T02:39:09.509755+0000 mon.smithi153 (mon.0) 1032 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:39:10.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:10 smithi153 bash[14340]: audit 2024-03-12T02:39:09.511320+0000 mon.smithi153 (mon.0) 1033 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:39:10.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:10 smithi153 bash[14340]: audit 2024-03-12T02:39:09.520126+0000 mon.smithi153 (mon.0) 1034 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:39:10.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:10 smithi153 bash[14340]: audit 2024-03-12T02:39:09.524417+0000 mon.smithi153 (mon.0) 1035 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:39:11.329 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:11 smithi153 bash[14340]: cluster 2024-03-12T02:39:09.513910+0000 mgr.smithi153.asomxr (mgr.14195) 1055 : cluster 0 pgmap v879: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:11.329 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:11 smithi153 bash[14340]: cluster 2024-03-12T02:39:09.514306+0000 mgr.smithi153.asomxr (mgr.14195) 1056 : cluster 0 pgmap v880: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:11 smithi163 bash[22347]: cluster 2024-03-12T02:39:09.513910+0000 mgr.smithi153.asomxr (mgr.14195) 1055 : cluster 0 pgmap v879: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:11 smithi163 bash[22347]: cluster 2024-03-12T02:39:09.514306+0000 mgr.smithi153.asomxr (mgr.14195) 1056 : cluster 0 pgmap v880: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:12.861 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:39:12.862 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:39:13.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:13 smithi153 bash[14340]: cluster 2024-03-12T02:39:11.514887+0000 mgr.smithi153.asomxr (mgr.14195) 1057 : cluster 0 pgmap v881: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:13.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:13 smithi163 bash[22347]: cluster 2024-03-12T02:39:11.514887+0000 mgr.smithi153.asomxr (mgr.14195) 1057 : cluster 0 pgmap v881: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:13.697 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:39:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:14 smithi163 bash[22347]: audit 2024-03-12T02:39:12.849589+0000 mgr.smithi153.asomxr (mgr.14195) 1058 : audit 0 from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:14.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:14 smithi153 bash[14340]: audit 2024-03-12T02:39:12.849589+0000 mgr.smithi153.asomxr (mgr.14195) 1058 : audit 0 from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:14.698 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:39:15.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:15 smithi163 bash[22347]: cluster 2024-03-12T02:39:13.515295+0000 mgr.smithi153.asomxr (mgr.14195) 1059 : cluster 0 pgmap v882: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:15.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:15 smithi153 bash[14340]: cluster 2024-03-12T02:39:13.515295+0000 mgr.smithi153.asomxr (mgr.14195) 1059 : cluster 0 pgmap v882: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:16 smithi163 bash[22347]: audit 2024-03-12T02:39:15.419160+0000 mon.smithi153 (mon.0) 1036 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:39:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:16 smithi153 bash[14340]: audit 2024-03-12T02:39:15.419160+0000 mon.smithi153 (mon.0) 1036 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:39:17.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:17 smithi163 bash[22347]: cluster 2024-03-12T02:39:15.515888+0000 mgr.smithi153.asomxr (mgr.14195) 1060 : cluster 0 pgmap v883: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:17.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:17 smithi153 bash[14340]: cluster 2024-03-12T02:39:15.515888+0000 mgr.smithi153.asomxr (mgr.14195) 1060 : cluster 0 pgmap v883: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:19.447 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:39:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:19 smithi163 bash[22347]: cluster 2024-03-12T02:39:17.516540+0000 mgr.smithi153.asomxr (mgr.14195) 1061 : cluster 0 pgmap v884: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:19 smithi153 bash[14340]: cluster 2024-03-12T02:39:17.516540+0000 mgr.smithi153.asomxr (mgr.14195) 1061 : cluster 0 pgmap v884: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:21 smithi163 bash[22347]: cluster 2024-03-12T02:39:19.516982+0000 mgr.smithi153.asomxr (mgr.14195) 1062 : cluster 0 pgmap v885: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:21.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:21 smithi153 bash[14340]: cluster 2024-03-12T02:39:19.516982+0000 mgr.smithi153.asomxr (mgr.14195) 1062 : cluster 0 pgmap v885: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:22.509 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:39:22.509 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:39:23.256 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:23 smithi153 bash[14340]: cluster 2024-03-12T02:39:21.517602+0000 mgr.smithi153.asomxr (mgr.14195) 1063 : cluster 0 pgmap v886: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:23.257 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:39:23.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:23 smithi163 bash[22347]: cluster 2024-03-12T02:39:21.517602+0000 mgr.smithi153.asomxr (mgr.14195) 1063 : cluster 0 pgmap v886: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:24.259 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:39:24.447 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:24 smithi153 bash[14340]: audit 2024-03-12T02:39:22.492897+0000 mgr.smithi153.asomxr (mgr.14195) 1064 : audit 0 from='client.15022 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:24 smithi163 bash[22347]: audit 2024-03-12T02:39:22.492897+0000 mgr.smithi153.asomxr (mgr.14195) 1064 : audit 0 from='client.15022 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:25.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:25 smithi163 bash[22347]: cluster 2024-03-12T02:39:23.518332+0000 mgr.smithi153.asomxr (mgr.14195) 1065 : cluster 0 pgmap v887: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:25.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:25 smithi153 bash[14340]: cluster 2024-03-12T02:39:23.518332+0000 mgr.smithi153.asomxr (mgr.14195) 1065 : cluster 0 pgmap v887: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:27.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:27 smithi163 bash[22347]: cluster 2024-03-12T02:39:25.518974+0000 mgr.smithi153.asomxr (mgr.14195) 1066 : cluster 0 pgmap v888: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:27.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:27 smithi153 bash[14340]: cluster 2024-03-12T02:39:25.518974+0000 mgr.smithi153.asomxr (mgr.14195) 1066 : cluster 0 pgmap v888: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:29.019 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:39:29.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:29 smithi153 bash[14340]: cluster 2024-03-12T02:39:27.519574+0000 mgr.smithi153.asomxr (mgr.14195) 1067 : cluster 0 pgmap v889: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:29 smithi163 bash[22347]: cluster 2024-03-12T02:39:27.519574+0000 mgr.smithi153.asomxr (mgr.14195) 1067 : cluster 0 pgmap v889: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:31 smithi163 bash[22347]: cluster 2024-03-12T02:39:29.520191+0000 mgr.smithi153.asomxr (mgr.14195) 1068 : cluster 0 pgmap v890: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:31.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:31 smithi163 bash[22347]: audit 2024-03-12T02:39:30.419498+0000 mon.smithi153 (mon.0) 1037 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:39:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:31 smithi153 bash[14340]: cluster 2024-03-12T02:39:29.520191+0000 mgr.smithi153.asomxr (mgr.14195) 1068 : cluster 0 pgmap v890: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:31 smithi153 bash[14340]: audit 2024-03-12T02:39:30.419498+0000 mon.smithi153 (mon.0) 1037 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:39:32.403 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:39:32.403 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:39:33.171 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:39:33.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:33 smithi153 bash[14340]: cluster 2024-03-12T02:39:31.520740+0000 mgr.smithi153.asomxr (mgr.14195) 1069 : cluster 0 pgmap v891: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:33.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:33 smithi163 bash[22347]: cluster 2024-03-12T02:39:31.520740+0000 mgr.smithi153.asomxr (mgr.14195) 1069 : cluster 0 pgmap v891: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:34.173 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:39:34.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:34 smithi163 bash[22347]: audit 2024-03-12T02:39:32.386983+0000 mgr.smithi153.asomxr (mgr.14195) 1070 : audit 0 from='client.15026 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:34.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:34 smithi153 bash[14340]: audit 2024-03-12T02:39:32.386983+0000 mgr.smithi153.asomxr (mgr.14195) 1070 : audit 0 from='client.15026 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:35.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:35 smithi163 bash[22347]: cluster 2024-03-12T02:39:33.521499+0000 mgr.smithi153.asomxr (mgr.14195) 1071 : cluster 0 pgmap v892: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:35.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:35 smithi153 bash[14340]: cluster 2024-03-12T02:39:33.521499+0000 mgr.smithi153.asomxr (mgr.14195) 1071 : cluster 0 pgmap v892: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:37.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:37 smithi163 bash[22347]: cluster 2024-03-12T02:39:35.522142+0000 mgr.smithi153.asomxr (mgr.14195) 1072 : cluster 0 pgmap v893: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:37.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:37 smithi153 bash[14340]: cluster 2024-03-12T02:39:35.522142+0000 mgr.smithi153.asomxr (mgr.14195) 1072 : cluster 0 pgmap v893: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:38.924 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:39:39.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:39 smithi153 bash[14340]: cluster 2024-03-12T02:39:37.522779+0000 mgr.smithi153.asomxr (mgr.14195) 1073 : cluster 0 pgmap v894: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:39.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:39 smithi163 bash[22347]: cluster 2024-03-12T02:39:37.522779+0000 mgr.smithi153.asomxr (mgr.14195) 1073 : cluster 0 pgmap v894: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:41.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:41 smithi153 bash[14340]: cluster 2024-03-12T02:39:39.523521+0000 mgr.smithi153.asomxr (mgr.14195) 1074 : cluster 0 pgmap v895: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:41.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:41 smithi163 bash[22347]: cluster 2024-03-12T02:39:39.523521+0000 mgr.smithi153.asomxr (mgr.14195) 1074 : cluster 0 pgmap v895: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:42.284 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:39:42.285 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:39:43.080 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:39:43.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:43 smithi153 bash[14340]: cluster 2024-03-12T02:39:41.524125+0000 mgr.smithi153.asomxr (mgr.14195) 1075 : cluster 0 pgmap v896: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:43.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:43 smithi163 bash[22347]: cluster 2024-03-12T02:39:41.524125+0000 mgr.smithi153.asomxr (mgr.14195) 1075 : cluster 0 pgmap v896: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:44.081 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:39:44.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:44 smithi163 bash[22347]: audit 2024-03-12T02:39:42.267847+0000 mgr.smithi153.asomxr (mgr.14195) 1076 : audit 0 from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:44.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:44 smithi153 bash[14340]: audit 2024-03-12T02:39:42.267847+0000 mgr.smithi153.asomxr (mgr.14195) 1076 : audit 0 from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:45.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:45 smithi163 bash[22347]: cluster 2024-03-12T02:39:43.524680+0000 mgr.smithi153.asomxr (mgr.14195) 1077 : cluster 0 pgmap v897: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:45.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:45 smithi153 bash[14340]: cluster 2024-03-12T02:39:43.524680+0000 mgr.smithi153.asomxr (mgr.14195) 1077 : cluster 0 pgmap v897: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:46 smithi163 bash[22347]: audit 2024-03-12T02:39:45.419781+0000 mon.smithi153 (mon.0) 1038 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:39:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:46 smithi163 bash[22347]: cluster 2024-03-12T02:39:45.525324+0000 mgr.smithi153.asomxr (mgr.14195) 1078 : cluster 0 pgmap v898: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:46 smithi153 bash[14340]: audit 2024-03-12T02:39:45.419781+0000 mon.smithi153 (mon.0) 1038 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:39:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:46 smithi153 bash[14340]: cluster 2024-03-12T02:39:45.525324+0000 mgr.smithi153.asomxr (mgr.14195) 1078 : cluster 0 pgmap v898: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:48.831 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:39:49.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:49 smithi153 bash[14340]: cluster 2024-03-12T02:39:47.525883+0000 mgr.smithi153.asomxr (mgr.14195) 1079 : cluster 0 pgmap v899: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:49.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:49 smithi163 bash[22347]: cluster 2024-03-12T02:39:47.525883+0000 mgr.smithi153.asomxr (mgr.14195) 1079 : cluster 0 pgmap v899: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:51.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:51 smithi153 bash[14340]: cluster 2024-03-12T02:39:49.526503+0000 mgr.smithi153.asomxr (mgr.14195) 1080 : cluster 0 pgmap v900: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:51 smithi163 bash[22347]: cluster 2024-03-12T02:39:49.526503+0000 mgr.smithi153.asomxr (mgr.14195) 1080 : cluster 0 pgmap v900: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:52.289 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:39:52.289 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:39:53.075 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:39:53.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:53 smithi153 bash[14340]: cluster 2024-03-12T02:39:51.527132+0000 mgr.smithi153.asomxr (mgr.14195) 1081 : cluster 0 pgmap v901: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:53.440 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:53 smithi163 bash[22347]: cluster 2024-03-12T02:39:51.527132+0000 mgr.smithi153.asomxr (mgr.14195) 1081 : cluster 0 pgmap v901: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:54.076 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:39:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:54 smithi163 bash[22347]: audit 2024-03-12T02:39:52.272850+0000 mgr.smithi153.asomxr (mgr.14195) 1082 : audit 0 from='client.15034 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:54.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:54 smithi153 bash[14340]: audit 2024-03-12T02:39:52.272850+0000 mgr.smithi153.asomxr (mgr.14195) 1082 : audit 0 from='client.15034 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:39:55.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:55 smithi153 bash[14340]: cluster 2024-03-12T02:39:53.527733+0000 mgr.smithi153.asomxr (mgr.14195) 1083 : cluster 0 pgmap v902: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:55.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:55 smithi163 bash[22347]: cluster 2024-03-12T02:39:53.527733+0000 mgr.smithi153.asomxr (mgr.14195) 1083 : cluster 0 pgmap v902: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:57.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:57 smithi163 bash[22347]: cluster 2024-03-12T02:39:55.528327+0000 mgr.smithi153.asomxr (mgr.14195) 1084 : cluster 0 pgmap v903: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:57.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:57 smithi153 bash[14340]: cluster 2024-03-12T02:39:55.528327+0000 mgr.smithi153.asomxr (mgr.14195) 1084 : cluster 0 pgmap v903: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:58.828 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:39:59.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:39:59 smithi153 bash[14340]: cluster 2024-03-12T02:39:57.528872+0000 mgr.smithi153.asomxr (mgr.14195) 1085 : cluster 0 pgmap v904: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:39:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:39:59 smithi163 bash[22347]: cluster 2024-03-12T02:39:57.528872+0000 mgr.smithi153.asomxr (mgr.14195) 1085 : cluster 0 pgmap v904: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:00.245 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:00 smithi153 bash[14340]: cluster 2024-03-12T02:40:00.000168+0000 mon.smithi153 (mon.0) 1039 : cluster 3 overall HEALTH_WARN 2 failed cephadm daemon(s) 2024-03-12T02:40:00.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:00 smithi163 bash[22347]: cluster 2024-03-12T02:40:00.000168+0000 mon.smithi153 (mon.0) 1039 : cluster 3 overall HEALTH_WARN 2 failed cephadm daemon(s) 2024-03-12T02:40:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:01 smithi153 bash[14340]: cluster 2024-03-12T02:39:59.529623+0000 mgr.smithi153.asomxr (mgr.14195) 1086 : cluster 0 pgmap v905: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:01 smithi153 bash[14340]: audit 2024-03-12T02:40:00.420161+0000 mon.smithi153 (mon.0) 1040 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:40:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:01 smithi163 bash[22347]: cluster 2024-03-12T02:39:59.529623+0000 mgr.smithi153.asomxr (mgr.14195) 1086 : cluster 0 pgmap v905: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:01 smithi163 bash[22347]: audit 2024-03-12T02:40:00.420161+0000 mon.smithi153 (mon.0) 1040 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:40:02.236 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:40:02.236 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:40:03.005 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:40:03.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:03 smithi153 bash[14340]: cluster 2024-03-12T02:40:01.530361+0000 mgr.smithi153.asomxr (mgr.14195) 1087 : cluster 0 pgmap v906: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:03.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:03 smithi163 bash[22347]: cluster 2024-03-12T02:40:01.530361+0000 mgr.smithi153.asomxr (mgr.14195) 1087 : cluster 0 pgmap v906: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:04.007 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:40:04.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:04 smithi153 bash[14340]: audit 2024-03-12T02:40:02.220077+0000 mgr.smithi153.asomxr (mgr.14195) 1088 : audit 0 from='client.15038 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:04 smithi163 bash[22347]: audit 2024-03-12T02:40:02.220077+0000 mgr.smithi153.asomxr (mgr.14195) 1088 : audit 0 from='client.15038 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:05.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:05 smithi163 bash[22347]: cluster 2024-03-12T02:40:03.530995+0000 mgr.smithi153.asomxr (mgr.14195) 1089 : cluster 0 pgmap v907: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:05.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:05 smithi153 bash[14340]: cluster 2024-03-12T02:40:03.530995+0000 mgr.smithi153.asomxr (mgr.14195) 1089 : cluster 0 pgmap v907: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:07 smithi163 bash[22347]: cluster 2024-03-12T02:40:05.531592+0000 mgr.smithi153.asomxr (mgr.14195) 1090 : cluster 0 pgmap v908: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:07.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:07 smithi153 bash[14340]: cluster 2024-03-12T02:40:05.531592+0000 mgr.smithi153.asomxr (mgr.14195) 1090 : cluster 0 pgmap v908: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:08.758 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:40:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:09 smithi163 bash[22347]: cluster 2024-03-12T02:40:07.532216+0000 mgr.smithi153.asomxr (mgr.14195) 1091 : cluster 0 pgmap v909: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:09.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:09 smithi153 bash[14340]: cluster 2024-03-12T02:40:07.532216+0000 mgr.smithi153.asomxr (mgr.14195) 1091 : cluster 0 pgmap v909: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:10.298 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:10 smithi153 bash[14340]: audit 2024-03-12T02:40:09.614613+0000 mon.smithi153 (mon.0) 1041 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:40:10.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:10 smithi163 bash[22347]: audit 2024-03-12T02:40:09.614613+0000 mon.smithi153 (mon.0) 1041 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:40:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:11 smithi153 bash[14340]: cluster 2024-03-12T02:40:09.532881+0000 mgr.smithi153.asomxr (mgr.14195) 1092 : cluster 0 pgmap v910: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:11 smithi153 bash[14340]: audit 2024-03-12T02:40:10.197010+0000 mon.smithi153 (mon.0) 1042 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:40:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:11 smithi153 bash[14340]: audit 2024-03-12T02:40:10.198755+0000 mon.smithi153 (mon.0) 1043 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:40:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:11 smithi153 bash[14340]: audit 2024-03-12T02:40:10.208506+0000 mon.smithi153 (mon.0) 1044 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:40:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:11 smithi153 bash[14340]: audit 2024-03-12T02:40:10.212861+0000 mon.smithi153 (mon.0) 1045 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:40:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:11 smithi163 bash[22347]: cluster 2024-03-12T02:40:09.532881+0000 mgr.smithi153.asomxr (mgr.14195) 1092 : cluster 0 pgmap v910: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:11 smithi163 bash[22347]: audit 2024-03-12T02:40:10.197010+0000 mon.smithi153 (mon.0) 1042 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:40:11.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:11 smithi163 bash[22347]: audit 2024-03-12T02:40:10.198755+0000 mon.smithi153 (mon.0) 1043 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:40:11.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:11 smithi163 bash[22347]: audit 2024-03-12T02:40:10.208506+0000 mon.smithi153 (mon.0) 1044 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:40:11.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:11 smithi163 bash[22347]: audit 2024-03-12T02:40:10.212861+0000 mon.smithi153 (mon.0) 1045 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:40:12.301 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:40:12.301 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:40:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:12 smithi163 bash[22347]: cluster 2024-03-12T02:40:10.201518+0000 mgr.smithi153.asomxr (mgr.14195) 1093 : cluster 0 pgmap v911: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:12 smithi163 bash[22347]: cluster 2024-03-12T02:40:10.201949+0000 mgr.smithi153.asomxr (mgr.14195) 1094 : cluster 0 pgmap v912: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:12.543 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:12 smithi153 bash[14340]: cluster 2024-03-12T02:40:10.201518+0000 mgr.smithi153.asomxr (mgr.14195) 1093 : cluster 0 pgmap v911: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:12.543 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:12 smithi153 bash[14340]: cluster 2024-03-12T02:40:10.201949+0000 mgr.smithi153.asomxr (mgr.14195) 1094 : cluster 0 pgmap v912: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:13.125 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:40:14.126 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:40:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:14 smithi163 bash[22347]: cluster 2024-03-12T02:40:12.202541+0000 mgr.smithi153.asomxr (mgr.14195) 1095 : cluster 0 pgmap v913: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:14 smithi163 bash[22347]: audit 2024-03-12T02:40:12.285273+0000 mgr.smithi153.asomxr (mgr.14195) 1096 : audit 0 from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:14.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:14 smithi153 bash[14340]: cluster 2024-03-12T02:40:12.202541+0000 mgr.smithi153.asomxr (mgr.14195) 1095 : cluster 0 pgmap v913: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:14.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:14 smithi153 bash[14340]: audit 2024-03-12T02:40:12.285273+0000 mgr.smithi153.asomxr (mgr.14195) 1096 : audit 0 from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:16 smithi163 bash[22347]: cluster 2024-03-12T02:40:14.203240+0000 mgr.smithi153.asomxr (mgr.14195) 1097 : cluster 0 pgmap v914: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:16 smithi163 bash[22347]: audit 2024-03-12T02:40:15.420235+0000 mon.smithi153 (mon.0) 1046 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:40:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:16 smithi153 bash[14340]: cluster 2024-03-12T02:40:14.203240+0000 mgr.smithi153.asomxr (mgr.14195) 1097 : cluster 0 pgmap v914: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:16 smithi153 bash[14340]: audit 2024-03-12T02:40:15.420235+0000 mon.smithi153 (mon.0) 1046 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:40:18.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:18 smithi163 bash[22347]: cluster 2024-03-12T02:40:16.203897+0000 mgr.smithi153.asomxr (mgr.14195) 1098 : cluster 0 pgmap v915: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:18.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:18 smithi153 bash[14340]: cluster 2024-03-12T02:40:16.203897+0000 mgr.smithi153.asomxr (mgr.14195) 1098 : cluster 0 pgmap v915: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:18.881 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:40:20.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:20 smithi153 bash[14340]: cluster 2024-03-12T02:40:18.204514+0000 mgr.smithi153.asomxr (mgr.14195) 1099 : cluster 0 pgmap v916: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:20 smithi163 bash[22347]: cluster 2024-03-12T02:40:18.204514+0000 mgr.smithi153.asomxr (mgr.14195) 1099 : cluster 0 pgmap v916: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:22.327 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:40:22.327 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:40:22.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:22 smithi163 bash[22347]: cluster 2024-03-12T02:40:20.205288+0000 mgr.smithi153.asomxr (mgr.14195) 1100 : cluster 0 pgmap v917: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:22.533 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:22 smithi153 bash[14340]: cluster 2024-03-12T02:40:20.205288+0000 mgr.smithi153.asomxr (mgr.14195) 1100 : cluster 0 pgmap v917: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:23.115 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:40:24.116 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:40:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:24 smithi163 bash[22347]: cluster 2024-03-12T02:40:22.205844+0000 mgr.smithi153.asomxr (mgr.14195) 1101 : cluster 0 pgmap v918: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:24 smithi163 bash[22347]: audit 2024-03-12T02:40:22.316215+0000 mgr.smithi153.asomxr (mgr.14195) 1102 : audit 0 from='client.15046 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:24.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:24 smithi153 bash[14340]: cluster 2024-03-12T02:40:22.205844+0000 mgr.smithi153.asomxr (mgr.14195) 1101 : cluster 0 pgmap v918: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:24.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:24 smithi153 bash[14340]: audit 2024-03-12T02:40:22.316215+0000 mgr.smithi153.asomxr (mgr.14195) 1102 : audit 0 from='client.15046 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:26 smithi163 bash[22347]: cluster 2024-03-12T02:40:24.206526+0000 mgr.smithi153.asomxr (mgr.14195) 1103 : cluster 0 pgmap v919: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:26 smithi153 bash[14340]: cluster 2024-03-12T02:40:24.206526+0000 mgr.smithi153.asomxr (mgr.14195) 1103 : cluster 0 pgmap v919: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:28 smithi163 bash[22347]: cluster 2024-03-12T02:40:26.207305+0000 mgr.smithi153.asomxr (mgr.14195) 1104 : cluster 0 pgmap v920: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:28.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:28 smithi153 bash[14340]: cluster 2024-03-12T02:40:26.207305+0000 mgr.smithi153.asomxr (mgr.14195) 1104 : cluster 0 pgmap v920: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:28.867 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:40:30.274 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:30 smithi153 bash[14340]: cluster 2024-03-12T02:40:28.208008+0000 mgr.smithi153.asomxr (mgr.14195) 1105 : cluster 0 pgmap v921: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:30.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:30 smithi163 bash[22347]: cluster 2024-03-12T02:40:28.208008+0000 mgr.smithi153.asomxr (mgr.14195) 1105 : cluster 0 pgmap v921: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:31.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:31 smithi153 bash[14340]: audit 2024-03-12T02:40:30.420377+0000 mon.smithi153 (mon.0) 1047 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:40:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:31 smithi163 bash[22347]: audit 2024-03-12T02:40:30.420377+0000 mon.smithi153 (mon.0) 1047 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:40:32.216 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:40:32.216 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:40:32.447 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:32 smithi153 bash[14340]: cluster 2024-03-12T02:40:30.208699+0000 mgr.smithi153.asomxr (mgr.14195) 1106 : cluster 0 pgmap v922: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:32 smithi163 bash[22347]: cluster 2024-03-12T02:40:30.208699+0000 mgr.smithi153.asomxr (mgr.14195) 1106 : cluster 0 pgmap v922: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:33.016 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:40:34.018 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:40:34.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:34 smithi153 bash[14340]: audit 2024-03-12T02:40:32.205964+0000 mgr.smithi153.asomxr (mgr.14195) 1107 : audit 0 from='client.15050 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:34.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:34 smithi153 bash[14340]: cluster 2024-03-12T02:40:32.209315+0000 mgr.smithi153.asomxr (mgr.14195) 1108 : cluster 0 pgmap v923: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:34.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:34 smithi163 bash[22347]: audit 2024-03-12T02:40:32.205964+0000 mgr.smithi153.asomxr (mgr.14195) 1107 : audit 0 from='client.15050 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:34.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:34 smithi163 bash[22347]: cluster 2024-03-12T02:40:32.209315+0000 mgr.smithi153.asomxr (mgr.14195) 1108 : cluster 0 pgmap v923: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:35.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:35 smithi153 bash[14340]: cluster 2024-03-12T02:40:34.209869+0000 mgr.smithi153.asomxr (mgr.14195) 1109 : cluster 0 pgmap v924: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:35.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:35 smithi163 bash[22347]: cluster 2024-03-12T02:40:34.209869+0000 mgr.smithi153.asomxr (mgr.14195) 1109 : cluster 0 pgmap v924: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:37.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:37 smithi153 bash[14340]: cluster 2024-03-12T02:40:36.210419+0000 mgr.smithi153.asomxr (mgr.14195) 1110 : cluster 0 pgmap v925: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:37.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:37 smithi163 bash[22347]: cluster 2024-03-12T02:40:36.210419+0000 mgr.smithi153.asomxr (mgr.14195) 1110 : cluster 0 pgmap v925: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:38.776 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:40:40.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:40 smithi163 bash[22347]: cluster 2024-03-12T02:40:38.211042+0000 mgr.smithi153.asomxr (mgr.14195) 1111 : cluster 0 pgmap v926: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:40.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:40 smithi153 bash[14340]: cluster 2024-03-12T02:40:38.211042+0000 mgr.smithi153.asomxr (mgr.14195) 1111 : cluster 0 pgmap v926: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:41.889 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:40:41.890 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:40:42.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:42 smithi153 bash[14340]: cluster 2024-03-12T02:40:40.211675+0000 mgr.smithi153.asomxr (mgr.14195) 1112 : cluster 0 pgmap v927: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:42.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:42 smithi163 bash[22347]: cluster 2024-03-12T02:40:40.211675+0000 mgr.smithi153.asomxr (mgr.14195) 1112 : cluster 0 pgmap v927: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:42.704 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:40:43.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:43 smithi153 bash[14340]: audit 2024-03-12T02:40:41.873988+0000 mgr.smithi153.asomxr (mgr.14195) 1113 : audit 0 from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:43.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:43 smithi163 bash[22347]: audit 2024-03-12T02:40:41.873988+0000 mgr.smithi153.asomxr (mgr.14195) 1113 : audit 0 from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:43.705 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:40:44.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:44 smithi163 bash[22347]: cluster 2024-03-12T02:40:42.212257+0000 mgr.smithi153.asomxr (mgr.14195) 1114 : cluster 0 pgmap v928: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:44.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:44 smithi153 bash[14340]: cluster 2024-03-12T02:40:42.212257+0000 mgr.smithi153.asomxr (mgr.14195) 1114 : cluster 0 pgmap v928: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:46 smithi153 bash[14340]: cluster 2024-03-12T02:40:44.212848+0000 mgr.smithi153.asomxr (mgr.14195) 1115 : cluster 0 pgmap v929: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:46 smithi153 bash[14340]: audit 2024-03-12T02:40:45.421382+0000 mon.smithi153 (mon.0) 1048 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:40:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:46 smithi163 bash[22347]: cluster 2024-03-12T02:40:44.212848+0000 mgr.smithi153.asomxr (mgr.14195) 1115 : cluster 0 pgmap v929: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:46 smithi163 bash[22347]: audit 2024-03-12T02:40:45.421382+0000 mon.smithi153 (mon.0) 1048 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:40:48.456 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:40:48.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:48 smithi163 bash[22347]: cluster 2024-03-12T02:40:46.213508+0000 mgr.smithi153.asomxr (mgr.14195) 1116 : cluster 0 pgmap v930: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:48.489 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:48 smithi153 bash[14340]: cluster 2024-03-12T02:40:46.213508+0000 mgr.smithi153.asomxr (mgr.14195) 1116 : cluster 0 pgmap v930: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:50 smithi163 bash[22347]: cluster 2024-03-12T02:40:48.214046+0000 mgr.smithi153.asomxr (mgr.14195) 1117 : cluster 0 pgmap v931: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:50.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:50 smithi153 bash[14340]: cluster 2024-03-12T02:40:48.214046+0000 mgr.smithi153.asomxr (mgr.14195) 1117 : cluster 0 pgmap v931: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:51.726 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:40:51.726 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:40:52.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:52 smithi153 bash[14340]: cluster 2024-03-12T02:40:50.214743+0000 mgr.smithi153.asomxr (mgr.14195) 1118 : cluster 0 pgmap v932: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:52 smithi163 bash[22347]: cluster 2024-03-12T02:40:50.214743+0000 mgr.smithi153.asomxr (mgr.14195) 1118 : cluster 0 pgmap v932: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:52.586 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:40:53.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:53 smithi153 bash[14340]: audit 2024-03-12T02:40:51.711184+0000 mgr.smithi153.asomxr (mgr.14195) 1119 : audit 0 from='client.15058 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:53.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:53 smithi163 bash[22347]: audit 2024-03-12T02:40:51.711184+0000 mgr.smithi153.asomxr (mgr.14195) 1119 : audit 0 from='client.15058 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:40:53.586 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:40:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:54 smithi163 bash[22347]: cluster 2024-03-12T02:40:52.215390+0000 mgr.smithi153.asomxr (mgr.14195) 1120 : cluster 0 pgmap v933: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:54.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:54 smithi153 bash[14340]: cluster 2024-03-12T02:40:52.215390+0000 mgr.smithi153.asomxr (mgr.14195) 1120 : cluster 0 pgmap v933: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:56.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:56 smithi163 bash[22347]: cluster 2024-03-12T02:40:54.215998+0000 mgr.smithi153.asomxr (mgr.14195) 1121 : cluster 0 pgmap v934: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:56.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:56 smithi153 bash[14340]: cluster 2024-03-12T02:40:54.215998+0000 mgr.smithi153.asomxr (mgr.14195) 1121 : cluster 0 pgmap v934: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:58.338 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:40:58.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:40:58 smithi163 bash[22347]: cluster 2024-03-12T02:40:56.216627+0000 mgr.smithi153.asomxr (mgr.14195) 1122 : cluster 0 pgmap v935: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:40:58.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:40:58 smithi153 bash[14340]: cluster 2024-03-12T02:40:56.216627+0000 mgr.smithi153.asomxr (mgr.14195) 1122 : cluster 0 pgmap v935: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:00.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:00 smithi163 bash[22347]: cluster 2024-03-12T02:40:58.217282+0000 mgr.smithi153.asomxr (mgr.14195) 1123 : cluster 0 pgmap v936: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:00.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:00 smithi153 bash[14340]: cluster 2024-03-12T02:40:58.217282+0000 mgr.smithi153.asomxr (mgr.14195) 1123 : cluster 0 pgmap v936: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:01 smithi163 bash[22347]: audit 2024-03-12T02:41:00.421082+0000 mon.smithi153 (mon.0) 1049 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:41:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:01 smithi153 bash[14340]: audit 2024-03-12T02:41:00.421082+0000 mon.smithi153 (mon.0) 1049 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:41:01.671 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:41:01.671 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:41:02.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:02 smithi153 bash[14340]: cluster 2024-03-12T02:41:00.217931+0000 mgr.smithi153.asomxr (mgr.14195) 1124 : cluster 0 pgmap v937: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:02 smithi163 bash[22347]: cluster 2024-03-12T02:41:00.217931+0000 mgr.smithi153.asomxr (mgr.14195) 1124 : cluster 0 pgmap v937: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:02.494 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:41:03.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:03 smithi153 bash[14340]: audit 2024-03-12T02:41:01.655824+0000 mgr.smithi153.asomxr (mgr.14195) 1125 : audit 0 from='client.15062 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:03.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:03 smithi163 bash[22347]: audit 2024-03-12T02:41:01.655824+0000 mgr.smithi153.asomxr (mgr.14195) 1125 : audit 0 from='client.15062 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:03.496 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:41:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:04 smithi163 bash[22347]: cluster 2024-03-12T02:41:02.218563+0000 mgr.smithi153.asomxr (mgr.14195) 1126 : cluster 0 pgmap v938: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:04.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:04 smithi153 bash[14340]: cluster 2024-03-12T02:41:02.218563+0000 mgr.smithi153.asomxr (mgr.14195) 1126 : cluster 0 pgmap v938: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:06.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:06 smithi163 bash[22347]: cluster 2024-03-12T02:41:04.219184+0000 mgr.smithi153.asomxr (mgr.14195) 1127 : cluster 0 pgmap v939: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:06.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:06 smithi153 bash[14340]: cluster 2024-03-12T02:41:04.219184+0000 mgr.smithi153.asomxr (mgr.14195) 1127 : cluster 0 pgmap v939: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:08.241 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:41:08.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:08 smithi163 bash[22347]: cluster 2024-03-12T02:41:06.219772+0000 mgr.smithi153.asomxr (mgr.14195) 1128 : cluster 0 pgmap v940: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:08.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:08 smithi153 bash[14340]: cluster 2024-03-12T02:41:06.219772+0000 mgr.smithi153.asomxr (mgr.14195) 1128 : cluster 0 pgmap v940: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:10.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:10 smithi163 bash[22347]: cluster 2024-03-12T02:41:08.220538+0000 mgr.smithi153.asomxr (mgr.14195) 1129 : cluster 0 pgmap v941: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:10.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:10 smithi153 bash[14340]: cluster 2024-03-12T02:41:08.220538+0000 mgr.smithi153.asomxr (mgr.14195) 1129 : cluster 0 pgmap v941: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:11.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:11 smithi153 bash[14340]: audit 2024-03-12T02:41:10.300322+0000 mon.smithi153 (mon.0) 1050 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:41:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:11 smithi153 bash[14340]: audit 2024-03-12T02:41:10.678545+0000 mon.smithi153 (mon.0) 1051 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:41:11.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:11 smithi153 bash[14340]: audit 2024-03-12T02:41:10.687043+0000 mon.smithi153 (mon.0) 1052 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:41:11.472 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:41:11.472 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:30:57.269932Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:30:57.269171Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:30:57.269444Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:30:57.270170Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:30:57.271584Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:30:57.271825Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:30:57.268926Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:30:57.268643Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:30:57.269692Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:30:57.270720Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:30:57.270472Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:41:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:11 smithi163 bash[22347]: audit 2024-03-12T02:41:10.300322+0000 mon.smithi153 (mon.0) 1050 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:41:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:11 smithi163 bash[22347]: audit 2024-03-12T02:41:10.678545+0000 mon.smithi153 (mon.0) 1051 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:41:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:11 smithi163 bash[22347]: audit 2024-03-12T02:41:10.687043+0000 mon.smithi153 (mon.0) 1052 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:41:12.261 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:41:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:12 smithi163 bash[22347]: cluster 2024-03-12T02:41:10.221050+0000 mgr.smithi153.asomxr (mgr.14195) 1130 : cluster 0 pgmap v942: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:12.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:12 smithi153 bash[14340]: cluster 2024-03-12T02:41:10.221050+0000 mgr.smithi153.asomxr (mgr.14195) 1130 : cluster 0 pgmap v942: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:13.262 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:41:13.273 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:13 smithi153 bash[14340]: audit 2024-03-12T02:41:11.456609+0000 mgr.smithi153.asomxr (mgr.14195) 1131 : audit 0 from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:13.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:13 smithi163 bash[22347]: audit 2024-03-12T02:41:11.456609+0000 mgr.smithi153.asomxr (mgr.14195) 1131 : audit 0 from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:14 smithi163 bash[22347]: cluster 2024-03-12T02:41:12.221559+0000 mgr.smithi153.asomxr (mgr.14195) 1132 : cluster 0 pgmap v943: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:14.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:14 smithi153 bash[14340]: cluster 2024-03-12T02:41:12.221559+0000 mgr.smithi153.asomxr (mgr.14195) 1132 : cluster 0 pgmap v943: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:16 smithi163 bash[22347]: cluster 2024-03-12T02:41:14.222230+0000 mgr.smithi153.asomxr (mgr.14195) 1133 : cluster 0 pgmap v944: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:16 smithi163 bash[22347]: audit 2024-03-12T02:41:15.422142+0000 mon.smithi153 (mon.0) 1053 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:41:16.534 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:16 smithi153 bash[14340]: cluster 2024-03-12T02:41:14.222230+0000 mgr.smithi153.asomxr (mgr.14195) 1133 : cluster 0 pgmap v944: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:16.535 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:16 smithi153 bash[14340]: audit 2024-03-12T02:41:15.422142+0000 mon.smithi153 (mon.0) 1053 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:41:17.836 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:41:18.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:18 smithi163 bash[22347]: cluster 2024-03-12T02:41:16.222657+0000 mgr.smithi153.asomxr (mgr.14195) 1134 : cluster 0 pgmap v945: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:18.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:18 smithi163 bash[22347]: audit 2024-03-12T02:41:17.353122+0000 mon.smithi153 (mon.0) 1054 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:41:18.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:18 smithi163 bash[22347]: audit 2024-03-12T02:41:17.362927+0000 mon.smithi153 (mon.0) 1055 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:41:18.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:18 smithi163 bash[22347]: audit 2024-03-12T02:41:17.887972+0000 mon.smithi153 (mon.0) 1056 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:41:18.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:18 smithi163 bash[22347]: audit 2024-03-12T02:41:17.889481+0000 mon.smithi153 (mon.0) 1057 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:41:18.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:18 smithi163 bash[22347]: audit 2024-03-12T02:41:17.897735+0000 mon.smithi153 (mon.0) 1058 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:41:18.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:18 smithi163 bash[22347]: audit 2024-03-12T02:41:17.902028+0000 mon.smithi153 (mon.0) 1059 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:41:18.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:18 smithi153 bash[14340]: cluster 2024-03-12T02:41:16.222657+0000 mgr.smithi153.asomxr (mgr.14195) 1134 : cluster 0 pgmap v945: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:18.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:18 smithi153 bash[14340]: audit 2024-03-12T02:41:17.353122+0000 mon.smithi153 (mon.0) 1054 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:41:18.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:18 smithi153 bash[14340]: audit 2024-03-12T02:41:17.362927+0000 mon.smithi153 (mon.0) 1055 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:41:18.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:18 smithi153 bash[14340]: audit 2024-03-12T02:41:17.887972+0000 mon.smithi153 (mon.0) 1056 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:41:18.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:18 smithi153 bash[14340]: audit 2024-03-12T02:41:17.889481+0000 mon.smithi153 (mon.0) 1057 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:41:18.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:18 smithi153 bash[14340]: audit 2024-03-12T02:41:17.897735+0000 mon.smithi153 (mon.0) 1058 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:41:18.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:18 smithi153 bash[14340]: audit 2024-03-12T02:41:17.902028+0000 mon.smithi153 (mon.0) 1059 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:41:19.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:19 smithi153 bash[14340]: cluster 2024-03-12T02:41:17.891798+0000 mgr.smithi153.asomxr (mgr.14195) 1135 : cluster 0 pgmap v946: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:19 smithi163 bash[22347]: cluster 2024-03-12T02:41:17.891798+0000 mgr.smithi153.asomxr (mgr.14195) 1135 : cluster 0 pgmap v946: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:21.303 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:41:21.304 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:41:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:21 smithi163 bash[22347]: cluster 2024-03-12T02:41:19.892288+0000 mgr.smithi153.asomxr (mgr.14195) 1136 : cluster 0 pgmap v947: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:21.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:21 smithi153 bash[14340]: cluster 2024-03-12T02:41:19.892288+0000 mgr.smithi153.asomxr (mgr.14195) 1136 : cluster 0 pgmap v947: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:22.070 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:41:23.071 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:41:23.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:23 smithi153 bash[14340]: audit 2024-03-12T02:41:21.286472+0000 mgr.smithi153.asomxr (mgr.14195) 1137 : audit 0 from='client.15070 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:23.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:23 smithi153 bash[14340]: cluster 2024-03-12T02:41:21.892900+0000 mgr.smithi153.asomxr (mgr.14195) 1138 : cluster 0 pgmap v948: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:23.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:23 smithi163 bash[22347]: audit 2024-03-12T02:41:21.286472+0000 mgr.smithi153.asomxr (mgr.14195) 1137 : audit 0 from='client.15070 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:23.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:23 smithi163 bash[22347]: cluster 2024-03-12T02:41:21.892900+0000 mgr.smithi153.asomxr (mgr.14195) 1138 : cluster 0 pgmap v948: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:24.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:24 smithi153 bash[14340]: cluster 2024-03-12T02:41:23.893561+0000 mgr.smithi153.asomxr (mgr.14195) 1139 : cluster 0 pgmap v949: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:24.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:24 smithi163 bash[22347]: cluster 2024-03-12T02:41:23.893561+0000 mgr.smithi153.asomxr (mgr.14195) 1139 : cluster 0 pgmap v949: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:27.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:26 smithi153 bash[14340]: cluster 2024-03-12T02:41:25.894164+0000 mgr.smithi153.asomxr (mgr.14195) 1140 : cluster 0 pgmap v950: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:27.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:26 smithi163 bash[22347]: cluster 2024-03-12T02:41:25.894164+0000 mgr.smithi153.asomxr (mgr.14195) 1140 : cluster 0 pgmap v950: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:27.822 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:41:29.181 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:29 smithi153 bash[14340]: cluster 2024-03-12T02:41:27.894826+0000 mgr.smithi153.asomxr (mgr.14195) 1141 : cluster 0 pgmap v951: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:29 smithi163 bash[22347]: cluster 2024-03-12T02:41:27.894826+0000 mgr.smithi153.asomxr (mgr.14195) 1141 : cluster 0 pgmap v951: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:31.149 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:41:31.150 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:41:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:31 smithi163 bash[22347]: cluster 2024-03-12T02:41:29.895297+0000 mgr.smithi153.asomxr (mgr.14195) 1142 : cluster 0 pgmap v952: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:31.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:31 smithi163 bash[22347]: audit 2024-03-12T02:41:30.422483+0000 mon.smithi153 (mon.0) 1060 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:41:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:31 smithi153 bash[14340]: cluster 2024-03-12T02:41:29.895297+0000 mgr.smithi153.asomxr (mgr.14195) 1142 : cluster 0 pgmap v952: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:31 smithi153 bash[14340]: audit 2024-03-12T02:41:30.422483+0000 mon.smithi153 (mon.0) 1060 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:41:31.867 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:41:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:32 smithi163 bash[22347]: audit 2024-03-12T02:41:31.139144+0000 mgr.smithi153.asomxr (mgr.14195) 1143 : audit 0 from='client.15074 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:32.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:32 smithi153 bash[14340]: audit 2024-03-12T02:41:31.139144+0000 mgr.smithi153.asomxr (mgr.14195) 1143 : audit 0 from='client.15074 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:32.868 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:41:33.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:33 smithi153 bash[14340]: cluster 2024-03-12T02:41:31.895865+0000 mgr.smithi153.asomxr (mgr.14195) 1144 : cluster 0 pgmap v953: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:33.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:33 smithi163 bash[22347]: cluster 2024-03-12T02:41:31.895865+0000 mgr.smithi153.asomxr (mgr.14195) 1144 : cluster 0 pgmap v953: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:35.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:35 smithi163 bash[22347]: cluster 2024-03-12T02:41:33.896520+0000 mgr.smithi153.asomxr (mgr.14195) 1145 : cluster 0 pgmap v954: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:35.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:35 smithi153 bash[14340]: cluster 2024-03-12T02:41:33.896520+0000 mgr.smithi153.asomxr (mgr.14195) 1145 : cluster 0 pgmap v954: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:37.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:37 smithi153 bash[14340]: cluster 2024-03-12T02:41:35.897161+0000 mgr.smithi153.asomxr (mgr.14195) 1146 : cluster 0 pgmap v955: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:37.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:37 smithi163 bash[22347]: cluster 2024-03-12T02:41:35.897161+0000 mgr.smithi153.asomxr (mgr.14195) 1146 : cluster 0 pgmap v955: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:37.626 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:41:39.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:39 smithi163 bash[22347]: cluster 2024-03-12T02:41:37.897623+0000 mgr.smithi153.asomxr (mgr.14195) 1147 : cluster 0 pgmap v956: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:39.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:39 smithi153 bash[14340]: cluster 2024-03-12T02:41:37.897623+0000 mgr.smithi153.asomxr (mgr.14195) 1147 : cluster 0 pgmap v956: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:41.067 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:41:41.067 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:41:41.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:41 smithi163 bash[22347]: cluster 2024-03-12T02:41:39.898103+0000 mgr.smithi153.asomxr (mgr.14195) 1148 : cluster 0 pgmap v957: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:41.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:41 smithi153 bash[14340]: cluster 2024-03-12T02:41:39.898103+0000 mgr.smithi153.asomxr (mgr.14195) 1148 : cluster 0 pgmap v957: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:41.875 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:41:42.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:42 smithi163 bash[22347]: audit 2024-03-12T02:41:41.050346+0000 mgr.smithi153.asomxr (mgr.14195) 1149 : audit 0 from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:42.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:42 smithi153 bash[14340]: audit 2024-03-12T02:41:41.050346+0000 mgr.smithi153.asomxr (mgr.14195) 1149 : audit 0 from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:42.876 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:41:43.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:43 smithi153 bash[14340]: cluster 2024-03-12T02:41:41.898724+0000 mgr.smithi153.asomxr (mgr.14195) 1150 : cluster 0 pgmap v958: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:43.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:43 smithi163 bash[22347]: cluster 2024-03-12T02:41:41.898724+0000 mgr.smithi153.asomxr (mgr.14195) 1150 : cluster 0 pgmap v958: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:45.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:45 smithi163 bash[22347]: cluster 2024-03-12T02:41:43.899463+0000 mgr.smithi153.asomxr (mgr.14195) 1151 : cluster 0 pgmap v959: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:45.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:45 smithi153 bash[14340]: cluster 2024-03-12T02:41:43.899463+0000 mgr.smithi153.asomxr (mgr.14195) 1151 : cluster 0 pgmap v959: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:46 smithi163 bash[22347]: audit 2024-03-12T02:41:45.422928+0000 mon.smithi153 (mon.0) 1061 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:41:46.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:46 smithi153 bash[14340]: audit 2024-03-12T02:41:45.422928+0000 mon.smithi153 (mon.0) 1061 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:41:47.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:47 smithi163 bash[22347]: cluster 2024-03-12T02:41:45.900205+0000 mgr.smithi153.asomxr (mgr.14195) 1152 : cluster 0 pgmap v960: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:47.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:47 smithi153 bash[14340]: cluster 2024-03-12T02:41:45.900205+0000 mgr.smithi153.asomxr (mgr.14195) 1152 : cluster 0 pgmap v960: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:47.623 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:41:49.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:49 smithi153 bash[14340]: cluster 2024-03-12T02:41:47.900878+0000 mgr.smithi153.asomxr (mgr.14195) 1153 : cluster 0 pgmap v961: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:49.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:49 smithi163 bash[22347]: cluster 2024-03-12T02:41:47.900878+0000 mgr.smithi153.asomxr (mgr.14195) 1153 : cluster 0 pgmap v961: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:50.885 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:41:50.885 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:41:51.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:51 smithi163 bash[22347]: cluster 2024-03-12T02:41:49.901514+0000 mgr.smithi153.asomxr (mgr.14195) 1154 : cluster 0 pgmap v962: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:51.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:51 smithi153 bash[14340]: cluster 2024-03-12T02:41:49.901514+0000 mgr.smithi153.asomxr (mgr.14195) 1154 : cluster 0 pgmap v962: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:51.688 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:41:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:52 smithi163 bash[22347]: audit 2024-03-12T02:41:50.868763+0000 mgr.smithi153.asomxr (mgr.14195) 1155 : audit 0 from='client.15082 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:52.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:52 smithi153 bash[14340]: audit 2024-03-12T02:41:50.868763+0000 mgr.smithi153.asomxr (mgr.14195) 1155 : audit 0 from='client.15082 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:41:52.690 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:41:53.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:53 smithi153 bash[14340]: cluster 2024-03-12T02:41:51.902130+0000 mgr.smithi153.asomxr (mgr.14195) 1156 : cluster 0 pgmap v963: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:53.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:53 smithi163 bash[22347]: cluster 2024-03-12T02:41:51.902130+0000 mgr.smithi153.asomxr (mgr.14195) 1156 : cluster 0 pgmap v963: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:55.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:55 smithi163 bash[22347]: cluster 2024-03-12T02:41:53.902697+0000 mgr.smithi153.asomxr (mgr.14195) 1157 : cluster 0 pgmap v964: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:55.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:55 smithi153 bash[14340]: cluster 2024-03-12T02:41:53.902697+0000 mgr.smithi153.asomxr (mgr.14195) 1157 : cluster 0 pgmap v964: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:57.440 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:41:57.471 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:57 smithi153 bash[14340]: cluster 2024-03-12T02:41:55.903203+0000 mgr.smithi153.asomxr (mgr.14195) 1158 : cluster 0 pgmap v965: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:57.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:57 smithi163 bash[22347]: cluster 2024-03-12T02:41:55.903203+0000 mgr.smithi153.asomxr (mgr.14195) 1158 : cluster 0 pgmap v965: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:41:59 smithi163 bash[22347]: cluster 2024-03-12T02:41:57.903812+0000 mgr.smithi153.asomxr (mgr.14195) 1159 : cluster 0 pgmap v966: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:41:59.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:41:59 smithi153 bash[14340]: cluster 2024-03-12T02:41:57.903812+0000 mgr.smithi153.asomxr (mgr.14195) 1159 : cluster 0 pgmap v966: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:00.618 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:42:00.619 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:42:01.441 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:01 smithi153 bash[14340]: cluster 2024-03-12T02:41:59.904467+0000 mgr.smithi153.asomxr (mgr.14195) 1160 : cluster 0 pgmap v967: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:01.441 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:01 smithi153 bash[14340]: audit 2024-03-12T02:42:00.423039+0000 mon.smithi153 (mon.0) 1062 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:42:01.442 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:42:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:01 smithi163 bash[22347]: cluster 2024-03-12T02:41:59.904467+0000 mgr.smithi153.asomxr (mgr.14195) 1160 : cluster 0 pgmap v967: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:01 smithi163 bash[22347]: audit 2024-03-12T02:42:00.423039+0000 mon.smithi153 (mon.0) 1062 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:42:02.443 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:42:02.457 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:02 smithi153 bash[14340]: audit 2024-03-12T02:42:00.604412+0000 mgr.smithi153.asomxr (mgr.14195) 1161 : audit 0 from='client.15086 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:02 smithi163 bash[22347]: audit 2024-03-12T02:42:00.604412+0000 mgr.smithi153.asomxr (mgr.14195) 1161 : audit 0 from='client.15086 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:03.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:03 smithi153 bash[14340]: cluster 2024-03-12T02:42:01.905158+0000 mgr.smithi153.asomxr (mgr.14195) 1162 : cluster 0 pgmap v968: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:03.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:03 smithi163 bash[22347]: cluster 2024-03-12T02:42:01.905158+0000 mgr.smithi153.asomxr (mgr.14195) 1162 : cluster 0 pgmap v968: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:05.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:05 smithi163 bash[22347]: cluster 2024-03-12T02:42:03.905764+0000 mgr.smithi153.asomxr (mgr.14195) 1163 : cluster 0 pgmap v969: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:05.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:05 smithi153 bash[14340]: cluster 2024-03-12T02:42:03.905764+0000 mgr.smithi153.asomxr (mgr.14195) 1163 : cluster 0 pgmap v969: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:07.194 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:42:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:07 smithi163 bash[22347]: cluster 2024-03-12T02:42:05.906452+0000 mgr.smithi153.asomxr (mgr.14195) 1164 : cluster 0 pgmap v970: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:07.530 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:07 smithi153 bash[14340]: cluster 2024-03-12T02:42:05.906452+0000 mgr.smithi153.asomxr (mgr.14195) 1164 : cluster 0 pgmap v970: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:09 smithi163 bash[22347]: cluster 2024-03-12T02:42:07.906986+0000 mgr.smithi153.asomxr (mgr.14195) 1165 : cluster 0 pgmap v971: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:09.487 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:09 smithi153 bash[14340]: cluster 2024-03-12T02:42:07.906986+0000 mgr.smithi153.asomxr (mgr.14195) 1165 : cluster 0 pgmap v971: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:10.611 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:42:10.611 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:42:11.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:11 smithi153 bash[14340]: cluster 2024-03-12T02:42:09.907692+0000 mgr.smithi153.asomxr (mgr.14195) 1166 : cluster 0 pgmap v972: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:11.480 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:42:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:11 smithi163 bash[22347]: cluster 2024-03-12T02:42:09.907692+0000 mgr.smithi153.asomxr (mgr.14195) 1166 : cluster 0 pgmap v972: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:12.481 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:42:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:12 smithi163 bash[22347]: audit 2024-03-12T02:42:10.593722+0000 mgr.smithi153.asomxr (mgr.14195) 1167 : audit 0 from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:12.495 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:12 smithi153 bash[14340]: audit 2024-03-12T02:42:10.593722+0000 mgr.smithi153.asomxr (mgr.14195) 1167 : audit 0 from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:13.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:13 smithi153 bash[14340]: cluster 2024-03-12T02:42:11.908339+0000 mgr.smithi153.asomxr (mgr.14195) 1168 : cluster 0 pgmap v973: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:13.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:13 smithi163 bash[22347]: cluster 2024-03-12T02:42:11.908339+0000 mgr.smithi153.asomxr (mgr.14195) 1168 : cluster 0 pgmap v973: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:15.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:15 smithi163 bash[22347]: cluster 2024-03-12T02:42:13.908980+0000 mgr.smithi153.asomxr (mgr.14195) 1169 : cluster 0 pgmap v974: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:15.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:15 smithi153 bash[14340]: cluster 2024-03-12T02:42:13.908980+0000 mgr.smithi153.asomxr (mgr.14195) 1169 : cluster 0 pgmap v974: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:16.403 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:16 smithi163 bash[22347]: audit 2024-03-12T02:42:15.423705+0000 mon.smithi153 (mon.0) 1063 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:42:16.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:16 smithi153 bash[14340]: audit 2024-03-12T02:42:15.423705+0000 mon.smithi153 (mon.0) 1063 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:42:17.236 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:42:17.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:17 smithi163 bash[22347]: cluster 2024-03-12T02:42:15.909660+0000 mgr.smithi153.asomxr (mgr.14195) 1170 : cluster 0 pgmap v975: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:17.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:17 smithi153 bash[14340]: cluster 2024-03-12T02:42:15.909660+0000 mgr.smithi153.asomxr (mgr.14195) 1170 : cluster 0 pgmap v975: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:18.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:18 smithi163 bash[22347]: audit 2024-03-12T02:42:17.974469+0000 mon.smithi153 (mon.0) 1064 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:42:18.513 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:18 smithi153 bash[14340]: audit 2024-03-12T02:42:17.974469+0000 mon.smithi153 (mon.0) 1064 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:42:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:19 smithi163 bash[22347]: cluster 2024-03-12T02:42:17.910222+0000 mgr.smithi153.asomxr (mgr.14195) 1171 : cluster 0 pgmap v976: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:19.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:19 smithi163 bash[22347]: audit 2024-03-12T02:42:18.829772+0000 mon.smithi153 (mon.0) 1065 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:42:19.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:19 smithi163 bash[22347]: audit 2024-03-12T02:42:18.833982+0000 mon.smithi153 (mon.0) 1066 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:42:19.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:19 smithi163 bash[22347]: audit 2024-03-12T02:42:18.840698+0000 mon.smithi153 (mon.0) 1067 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:42:19.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:19 smithi163 bash[22347]: audit 2024-03-12T02:42:18.845420+0000 mon.smithi153 (mon.0) 1068 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:42:19.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:19 smithi163 bash[22347]: audit 2024-03-12T02:42:18.846856+0000 mon.smithi153 (mon.0) 1069 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:42:19.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:19 smithi163 bash[22347]: audit 2024-03-12T02:42:18.847808+0000 mon.smithi153 (mon.0) 1070 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:42:19.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:19 smithi163 bash[22347]: audit 2024-03-12T02:42:18.854585+0000 mon.smithi153 (mon.0) 1071 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:42:19.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:19 smithi163 bash[22347]: audit 2024-03-12T02:42:18.858345+0000 mon.smithi153 (mon.0) 1072 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:42:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:19 smithi153 bash[14340]: cluster 2024-03-12T02:42:17.910222+0000 mgr.smithi153.asomxr (mgr.14195) 1171 : cluster 0 pgmap v976: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:19 smithi153 bash[14340]: audit 2024-03-12T02:42:18.829772+0000 mon.smithi153 (mon.0) 1065 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:42:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:19 smithi153 bash[14340]: audit 2024-03-12T02:42:18.833982+0000 mon.smithi153 (mon.0) 1066 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:42:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:19 smithi153 bash[14340]: audit 2024-03-12T02:42:18.840698+0000 mon.smithi153 (mon.0) 1067 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:42:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:19 smithi153 bash[14340]: audit 2024-03-12T02:42:18.845420+0000 mon.smithi153 (mon.0) 1068 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:42:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:19 smithi153 bash[14340]: audit 2024-03-12T02:42:18.846856+0000 mon.smithi153 (mon.0) 1069 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:42:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:19 smithi153 bash[14340]: audit 2024-03-12T02:42:18.847808+0000 mon.smithi153 (mon.0) 1070 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:42:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:19 smithi153 bash[14340]: audit 2024-03-12T02:42:18.854585+0000 mon.smithi153 (mon.0) 1071 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:42:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:19 smithi153 bash[14340]: audit 2024-03-12T02:42:18.858345+0000 mon.smithi153 (mon.0) 1072 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:42:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:20 smithi163 bash[22347]: cluster 2024-03-12T02:42:18.849657+0000 mgr.smithi153.asomxr (mgr.14195) 1172 : cluster 0 pgmap v977: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:20 smithi163 bash[22347]: cluster 2024-03-12T02:42:18.850037+0000 mgr.smithi153.asomxr (mgr.14195) 1173 : cluster 0 pgmap v978: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:20.556 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:42:20.556 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:42:20.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:20 smithi153 bash[14340]: cluster 2024-03-12T02:42:18.849657+0000 mgr.smithi153.asomxr (mgr.14195) 1172 : cluster 0 pgmap v977: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:20.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:20 smithi153 bash[14340]: cluster 2024-03-12T02:42:18.850037+0000 mgr.smithi153.asomxr (mgr.14195) 1173 : cluster 0 pgmap v978: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:21.405 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:42:22.406 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:42:22.419 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:22 smithi153 bash[14340]: audit 2024-03-12T02:42:20.540521+0000 mgr.smithi153.asomxr (mgr.14195) 1174 : audit 0 from='client.15094 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:22.419 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:22 smithi153 bash[14340]: cluster 2024-03-12T02:42:20.850641+0000 mgr.smithi153.asomxr (mgr.14195) 1175 : cluster 0 pgmap v979: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:22.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:22 smithi163 bash[22347]: audit 2024-03-12T02:42:20.540521+0000 mgr.smithi153.asomxr (mgr.14195) 1174 : audit 0 from='client.15094 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:22.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:22 smithi163 bash[22347]: cluster 2024-03-12T02:42:20.850641+0000 mgr.smithi153.asomxr (mgr.14195) 1175 : cluster 0 pgmap v979: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:24 smithi163 bash[22347]: cluster 2024-03-12T02:42:22.851260+0000 mgr.smithi153.asomxr (mgr.14195) 1176 : cluster 0 pgmap v980: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:24.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:24 smithi153 bash[14340]: cluster 2024-03-12T02:42:22.851260+0000 mgr.smithi153.asomxr (mgr.14195) 1176 : cluster 0 pgmap v980: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:26 smithi163 bash[22347]: cluster 2024-03-12T02:42:24.851896+0000 mgr.smithi153.asomxr (mgr.14195) 1177 : cluster 0 pgmap v981: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:26.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:26 smithi153 bash[14340]: cluster 2024-03-12T02:42:24.851896+0000 mgr.smithi153.asomxr (mgr.14195) 1177 : cluster 0 pgmap v981: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:27.160 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:42:28.428 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:28 smithi153 bash[14340]: cluster 2024-03-12T02:42:26.852458+0000 mgr.smithi153.asomxr (mgr.14195) 1178 : cluster 0 pgmap v982: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:28 smithi163 bash[22347]: cluster 2024-03-12T02:42:26.852458+0000 mgr.smithi153.asomxr (mgr.14195) 1178 : cluster 0 pgmap v982: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:30.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:30 smithi153 bash[14340]: cluster 2024-03-12T02:42:28.853075+0000 mgr.smithi153.asomxr (mgr.14195) 1179 : cluster 0 pgmap v983: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:30.434 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:42:30.434 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:42:30.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:30 smithi163 bash[22347]: cluster 2024-03-12T02:42:28.853075+0000 mgr.smithi153.asomxr (mgr.14195) 1179 : cluster 0 pgmap v983: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:31.290 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:42:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:31 smithi163 bash[22347]: audit 2024-03-12T02:42:30.426066+0000 mon.smithi153 (mon.0) 1073 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:42:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:31 smithi153 bash[14340]: audit 2024-03-12T02:42:30.426066+0000 mon.smithi153 (mon.0) 1073 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:42:32.291 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:42:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:32 smithi163 bash[22347]: audit 2024-03-12T02:42:30.416738+0000 mgr.smithi153.asomxr (mgr.14195) 1180 : audit 0 from='client.15098 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:32 smithi163 bash[22347]: cluster 2024-03-12T02:42:30.853489+0000 mgr.smithi153.asomxr (mgr.14195) 1181 : cluster 0 pgmap v984: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:32.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:32 smithi153 bash[14340]: audit 2024-03-12T02:42:30.416738+0000 mgr.smithi153.asomxr (mgr.14195) 1180 : audit 0 from='client.15098 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:32.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:32 smithi153 bash[14340]: cluster 2024-03-12T02:42:30.853489+0000 mgr.smithi153.asomxr (mgr.14195) 1181 : cluster 0 pgmap v984: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:34.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:34 smithi163 bash[22347]: cluster 2024-03-12T02:42:32.853979+0000 mgr.smithi153.asomxr (mgr.14195) 1182 : cluster 0 pgmap v985: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:34.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:34 smithi153 bash[14340]: cluster 2024-03-12T02:42:32.853979+0000 mgr.smithi153.asomxr (mgr.14195) 1182 : cluster 0 pgmap v985: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:36.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:36 smithi163 bash[22347]: cluster 2024-03-12T02:42:34.854577+0000 mgr.smithi153.asomxr (mgr.14195) 1183 : cluster 0 pgmap v986: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:36.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:36 smithi153 bash[14340]: cluster 2024-03-12T02:42:34.854577+0000 mgr.smithi153.asomxr (mgr.14195) 1183 : cluster 0 pgmap v986: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:37.040 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:42:38.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:38 smithi153 bash[14340]: cluster 2024-03-12T02:42:36.855156+0000 mgr.smithi153.asomxr (mgr.14195) 1184 : cluster 0 pgmap v987: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:38.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:38 smithi163 bash[22347]: cluster 2024-03-12T02:42:36.855156+0000 mgr.smithi153.asomxr (mgr.14195) 1184 : cluster 0 pgmap v987: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:40.200 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:42:40.200 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:42:40.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:40 smithi163 bash[22347]: cluster 2024-03-12T02:42:38.855701+0000 mgr.smithi153.asomxr (mgr.14195) 1185 : cluster 0 pgmap v988: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:40.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:40 smithi153 bash[14340]: cluster 2024-03-12T02:42:38.855701+0000 mgr.smithi153.asomxr (mgr.14195) 1185 : cluster 0 pgmap v988: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:41.069 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:42:41.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:41 smithi153 bash[14340]: audit 2024-03-12T02:42:40.185794+0000 mgr.smithi153.asomxr (mgr.14195) 1186 : audit 0 from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:41.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:41 smithi163 bash[22347]: audit 2024-03-12T02:42:40.185794+0000 mgr.smithi153.asomxr (mgr.14195) 1186 : audit 0 from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:42.069 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:42:42.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:42 smithi163 bash[22347]: cluster 2024-03-12T02:42:40.856336+0000 mgr.smithi153.asomxr (mgr.14195) 1187 : cluster 0 pgmap v989: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:42.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:42 smithi153 bash[14340]: cluster 2024-03-12T02:42:40.856336+0000 mgr.smithi153.asomxr (mgr.14195) 1187 : cluster 0 pgmap v989: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:44.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:44 smithi163 bash[22347]: cluster 2024-03-12T02:42:42.856912+0000 mgr.smithi153.asomxr (mgr.14195) 1188 : cluster 0 pgmap v990: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:44.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:44 smithi153 bash[14340]: cluster 2024-03-12T02:42:42.856912+0000 mgr.smithi153.asomxr (mgr.14195) 1188 : cluster 0 pgmap v990: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:46 smithi163 bash[22347]: cluster 2024-03-12T02:42:44.857561+0000 mgr.smithi153.asomxr (mgr.14195) 1189 : cluster 0 pgmap v991: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:46 smithi163 bash[22347]: audit 2024-03-12T02:42:45.427009+0000 mon.smithi153 (mon.0) 1074 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:42:46.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:46 smithi153 bash[14340]: cluster 2024-03-12T02:42:44.857561+0000 mgr.smithi153.asomxr (mgr.14195) 1189 : cluster 0 pgmap v991: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:46 smithi153 bash[14340]: audit 2024-03-12T02:42:45.427009+0000 mon.smithi153 (mon.0) 1074 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:42:46.824 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:42:48.217 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:48 smithi153 bash[14340]: cluster 2024-03-12T02:42:46.858117+0000 mgr.smithi153.asomxr (mgr.14195) 1190 : cluster 0 pgmap v992: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:48.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:48 smithi163 bash[22347]: cluster 2024-03-12T02:42:46.858117+0000 mgr.smithi153.asomxr (mgr.14195) 1190 : cluster 0 pgmap v992: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:50.207 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:42:50.208 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:42:50.450 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:50 smithi153 bash[14340]: cluster 2024-03-12T02:42:48.858714+0000 mgr.smithi153.asomxr (mgr.14195) 1191 : cluster 0 pgmap v993: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:50 smithi163 bash[22347]: cluster 2024-03-12T02:42:48.858714+0000 mgr.smithi153.asomxr (mgr.14195) 1191 : cluster 0 pgmap v993: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:50.996 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:42:51.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:51 smithi153 bash[14340]: audit 2024-03-12T02:42:50.190460+0000 mgr.smithi153.asomxr (mgr.14195) 1192 : audit 0 from='client.15106 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:51 smithi163 bash[22347]: audit 2024-03-12T02:42:50.190460+0000 mgr.smithi153.asomxr (mgr.14195) 1192 : audit 0 from='client.15106 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:42:51.997 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:42:52.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:52 smithi153 bash[14340]: cluster 2024-03-12T02:42:50.859330+0000 mgr.smithi153.asomxr (mgr.14195) 1193 : cluster 0 pgmap v994: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:52 smithi163 bash[22347]: cluster 2024-03-12T02:42:50.859330+0000 mgr.smithi153.asomxr (mgr.14195) 1193 : cluster 0 pgmap v994: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:54.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:54 smithi163 bash[22347]: cluster 2024-03-12T02:42:52.859905+0000 mgr.smithi153.asomxr (mgr.14195) 1194 : cluster 0 pgmap v995: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:54.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:54 smithi153 bash[14340]: cluster 2024-03-12T02:42:52.859905+0000 mgr.smithi153.asomxr (mgr.14195) 1194 : cluster 0 pgmap v995: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:56.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:56 smithi163 bash[22347]: cluster 2024-03-12T02:42:54.860508+0000 mgr.smithi153.asomxr (mgr.14195) 1195 : cluster 0 pgmap v996: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:56.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:56 smithi153 bash[14340]: cluster 2024-03-12T02:42:54.860508+0000 mgr.smithi153.asomxr (mgr.14195) 1195 : cluster 0 pgmap v996: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:56.750 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:42:58.416 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:42:58 smithi153 bash[14340]: cluster 2024-03-12T02:42:56.861160+0000 mgr.smithi153.asomxr (mgr.14195) 1196 : cluster 0 pgmap v997: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:42:58.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:42:58 smithi163 bash[22347]: cluster 2024-03-12T02:42:56.861160+0000 mgr.smithi153.asomxr (mgr.14195) 1196 : cluster 0 pgmap v997: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:00.158 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:43:00.159 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:43:00.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:00 smithi153 bash[14340]: cluster 2024-03-12T02:42:58.861799+0000 mgr.smithi153.asomxr (mgr.14195) 1197 : cluster 0 pgmap v998: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:00.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:00 smithi163 bash[22347]: cluster 2024-03-12T02:42:58.861799+0000 mgr.smithi153.asomxr (mgr.14195) 1197 : cluster 0 pgmap v998: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:01.009 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:43:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:01 smithi153 bash[14340]: audit 2024-03-12T02:43:00.146670+0000 mgr.smithi153.asomxr (mgr.14195) 1198 : audit 0 from='client.15110 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:01 smithi153 bash[14340]: audit 2024-03-12T02:43:00.426827+0000 mon.smithi153 (mon.0) 1075 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:43:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:01 smithi163 bash[22347]: audit 2024-03-12T02:43:00.146670+0000 mgr.smithi153.asomxr (mgr.14195) 1198 : audit 0 from='client.15110 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:01 smithi163 bash[22347]: audit 2024-03-12T02:43:00.426827+0000 mon.smithi153 (mon.0) 1075 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:43:02.010 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:43:02.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:02 smithi153 bash[14340]: cluster 2024-03-12T02:43:00.862368+0000 mgr.smithi153.asomxr (mgr.14195) 1199 : cluster 0 pgmap v999: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:02 smithi163 bash[22347]: cluster 2024-03-12T02:43:00.862368+0000 mgr.smithi153.asomxr (mgr.14195) 1199 : cluster 0 pgmap v999: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:04 smithi163 bash[22347]: cluster 2024-03-12T02:43:02.862996+0000 mgr.smithi153.asomxr (mgr.14195) 1200 : cluster 0 pgmap v1000: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:04.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:04 smithi153 bash[14340]: cluster 2024-03-12T02:43:02.862996+0000 mgr.smithi153.asomxr (mgr.14195) 1200 : cluster 0 pgmap v1000: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:06.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:06 smithi163 bash[22347]: cluster 2024-03-12T02:43:04.863623+0000 mgr.smithi153.asomxr (mgr.14195) 1201 : cluster 0 pgmap v1001: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:06.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:06 smithi153 bash[14340]: cluster 2024-03-12T02:43:04.863623+0000 mgr.smithi153.asomxr (mgr.14195) 1201 : cluster 0 pgmap v1001: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:06.767 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:43:08.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:08 smithi163 bash[22347]: cluster 2024-03-12T02:43:06.864307+0000 mgr.smithi153.asomxr (mgr.14195) 1202 : cluster 0 pgmap v1002: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:08.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:08 smithi153 bash[14340]: cluster 2024-03-12T02:43:06.864307+0000 mgr.smithi153.asomxr (mgr.14195) 1202 : cluster 0 pgmap v1002: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:10.135 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:43:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:43:10.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:10 smithi163 bash[22347]: cluster 2024-03-12T02:43:08.864901+0000 mgr.smithi153.asomxr (mgr.14195) 1203 : cluster 0 pgmap v1003: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:10.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:10 smithi153 bash[14340]: cluster 2024-03-12T02:43:08.864901+0000 mgr.smithi153.asomxr (mgr.14195) 1203 : cluster 0 pgmap v1003: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:10.967 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:43:11.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:11 smithi153 bash[14340]: audit 2024-03-12T02:43:10.120275+0000 mgr.smithi153.asomxr (mgr.14195) 1204 : audit 0 from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:11 smithi163 bash[22347]: audit 2024-03-12T02:43:10.120275+0000 mgr.smithi153.asomxr (mgr.14195) 1204 : audit 0 from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:11.968 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:43:12.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:12 smithi153 bash[14340]: cluster 2024-03-12T02:43:10.865437+0000 mgr.smithi153.asomxr (mgr.14195) 1205 : cluster 0 pgmap v1004: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:12.485 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:12 smithi163 bash[22347]: cluster 2024-03-12T02:43:10.865437+0000 mgr.smithi153.asomxr (mgr.14195) 1205 : cluster 0 pgmap v1004: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:14 smithi163 bash[22347]: cluster 2024-03-12T02:43:12.866031+0000 mgr.smithi153.asomxr (mgr.14195) 1206 : cluster 0 pgmap v1005: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:14.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:14 smithi153 bash[14340]: cluster 2024-03-12T02:43:12.866031+0000 mgr.smithi153.asomxr (mgr.14195) 1206 : cluster 0 pgmap v1005: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:16 smithi163 bash[22347]: cluster 2024-03-12T02:43:14.866680+0000 mgr.smithi153.asomxr (mgr.14195) 1207 : cluster 0 pgmap v1006: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:16 smithi163 bash[22347]: audit 2024-03-12T02:43:15.427696+0000 mon.smithi153 (mon.0) 1076 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:43:16.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:16 smithi153 bash[14340]: cluster 2024-03-12T02:43:14.866680+0000 mgr.smithi153.asomxr (mgr.14195) 1207 : cluster 0 pgmap v1006: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:16 smithi153 bash[14340]: audit 2024-03-12T02:43:15.427696+0000 mon.smithi153 (mon.0) 1076 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:43:16.729 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:43:18.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:18 smithi163 bash[22347]: cluster 2024-03-12T02:43:16.867318+0000 mgr.smithi153.asomxr (mgr.14195) 1208 : cluster 0 pgmap v1007: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:18.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:18 smithi153 bash[14340]: cluster 2024-03-12T02:43:16.867318+0000 mgr.smithi153.asomxr (mgr.14195) 1208 : cluster 0 pgmap v1007: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:19 smithi163 bash[22347]: audit 2024-03-12T02:43:18.949306+0000 mon.smithi153 (mon.0) 1077 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:43:19.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:19 smithi153 bash[14340]: audit 2024-03-12T02:43:18.949306+0000 mon.smithi153 (mon.0) 1077 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:43:20.212 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:43:20.213 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:33:04.804442Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:33:04.804749Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:33:04.806906Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:33:04.806663Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:33:04.807150Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:33:04.805293Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:33:04.805544Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:33:04.805010Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:33:04.805788Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:43:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:20 smithi163 bash[22347]: cluster 2024-03-12T02:43:18.867758+0000 mgr.smithi153.asomxr (mgr.14195) 1209 : cluster 0 pgmap v1008: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:20.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:20 smithi163 bash[22347]: audit 2024-03-12T02:43:19.325155+0000 mon.smithi153 (mon.0) 1078 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:43:20.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:20 smithi163 bash[22347]: audit 2024-03-12T02:43:19.333885+0000 mon.smithi153 (mon.0) 1079 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:43:20.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:20 smithi163 bash[22347]: audit 2024-03-12T02:43:20.197221+0000 mgr.smithi153.asomxr (mgr.14195) 1210 : audit 0 from='client.15118 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:20.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:20 smithi153 bash[14340]: cluster 2024-03-12T02:43:18.867758+0000 mgr.smithi153.asomxr (mgr.14195) 1209 : cluster 0 pgmap v1008: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:20.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:20 smithi153 bash[14340]: audit 2024-03-12T02:43:19.325155+0000 mon.smithi153 (mon.0) 1078 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:43:20.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:20 smithi153 bash[14340]: audit 2024-03-12T02:43:19.333885+0000 mon.smithi153 (mon.0) 1079 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:43:20.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:20 smithi153 bash[14340]: audit 2024-03-12T02:43:20.197221+0000 mgr.smithi153.asomxr (mgr.14195) 1210 : audit 0 from='client.15118 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:21.013 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:43:21.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:21 smithi153 bash[14340]: cluster 2024-03-12T02:43:20.868349+0000 mgr.smithi153.asomxr (mgr.14195) 1211 : cluster 0 pgmap v1009: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:21.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:21 smithi163 bash[22347]: cluster 2024-03-12T02:43:20.868349+0000 mgr.smithi153.asomxr (mgr.14195) 1211 : cluster 0 pgmap v1009: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:22.014 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:43:24.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:24 smithi153 bash[14340]: cluster 2024-03-12T02:43:22.869004+0000 mgr.smithi153.asomxr (mgr.14195) 1212 : cluster 0 pgmap v1010: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:24 smithi163 bash[22347]: cluster 2024-03-12T02:43:22.869004+0000 mgr.smithi153.asomxr (mgr.14195) 1212 : cluster 0 pgmap v1010: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:26 smithi163 bash[22347]: cluster 2024-03-12T02:43:24.869695+0000 mgr.smithi153.asomxr (mgr.14195) 1213 : cluster 0 pgmap v1011: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:26.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:26 smithi163 bash[22347]: audit 2024-03-12T02:43:25.252135+0000 mon.smithi153 (mon.0) 1080 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:43:26.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:26 smithi163 bash[22347]: audit 2024-03-12T02:43:25.260119+0000 mon.smithi153 (mon.0) 1081 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:43:26.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:26 smithi163 bash[22347]: audit 2024-03-12T02:43:25.780853+0000 mon.smithi153 (mon.0) 1082 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:43:26.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:26 smithi163 bash[22347]: audit 2024-03-12T02:43:25.782622+0000 mon.smithi153 (mon.0) 1083 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:43:26.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:26 smithi163 bash[22347]: audit 2024-03-12T02:43:25.792359+0000 mon.smithi153 (mon.0) 1084 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:43:26.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:26 smithi163 bash[22347]: audit 2024-03-12T02:43:25.798225+0000 mon.smithi153 (mon.0) 1085 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:43:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:26 smithi153 bash[14340]: cluster 2024-03-12T02:43:24.869695+0000 mgr.smithi153.asomxr (mgr.14195) 1213 : cluster 0 pgmap v1011: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:26 smithi153 bash[14340]: audit 2024-03-12T02:43:25.252135+0000 mon.smithi153 (mon.0) 1080 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:43:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:26 smithi153 bash[14340]: audit 2024-03-12T02:43:25.260119+0000 mon.smithi153 (mon.0) 1081 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:43:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:26 smithi153 bash[14340]: audit 2024-03-12T02:43:25.780853+0000 mon.smithi153 (mon.0) 1082 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:43:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:26 smithi153 bash[14340]: audit 2024-03-12T02:43:25.782622+0000 mon.smithi153 (mon.0) 1083 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:43:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:26 smithi153 bash[14340]: audit 2024-03-12T02:43:25.792359+0000 mon.smithi153 (mon.0) 1084 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:43:26.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:26 smithi153 bash[14340]: audit 2024-03-12T02:43:25.798225+0000 mon.smithi153 (mon.0) 1085 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:43:26.764 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:43:27.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:27 smithi163 bash[22347]: cluster 2024-03-12T02:43:25.785662+0000 mgr.smithi153.asomxr (mgr.14195) 1214 : cluster 0 pgmap v1012: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:27.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:27 smithi163 bash[22347]: cluster 2024-03-12T02:43:25.786177+0000 mgr.smithi153.asomxr (mgr.14195) 1215 : cluster 0 pgmap v1013: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:27.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:27 smithi153 bash[14340]: cluster 2024-03-12T02:43:25.785662+0000 mgr.smithi153.asomxr (mgr.14195) 1214 : cluster 0 pgmap v1012: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:27.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:27 smithi153 bash[14340]: cluster 2024-03-12T02:43:25.786177+0000 mgr.smithi153.asomxr (mgr.14195) 1215 : cluster 0 pgmap v1013: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:29.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:29 smithi153 bash[14340]: cluster 2024-03-12T02:43:27.786863+0000 mgr.smithi153.asomxr (mgr.14195) 1216 : cluster 0 pgmap v1014: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:29 smithi163 bash[22347]: cluster 2024-03-12T02:43:27.786863+0000 mgr.smithi153.asomxr (mgr.14195) 1216 : cluster 0 pgmap v1014: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:29.981 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:43:29.981 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:43:30.785 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:43:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:31 smithi163 bash[22347]: cluster 2024-03-12T02:43:29.787346+0000 mgr.smithi153.asomxr (mgr.14195) 1217 : cluster 0 pgmap v1015: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:31 smithi163 bash[22347]: audit 2024-03-12T02:43:29.964969+0000 mgr.smithi153.asomxr (mgr.14195) 1218 : audit 0 from='client.15122 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:31 smithi163 bash[22347]: audit 2024-03-12T02:43:30.427983+0000 mon.smithi153 (mon.0) 1086 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:43:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:31 smithi153 bash[14340]: cluster 2024-03-12T02:43:29.787346+0000 mgr.smithi153.asomxr (mgr.14195) 1217 : cluster 0 pgmap v1015: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:31 smithi153 bash[14340]: audit 2024-03-12T02:43:29.964969+0000 mgr.smithi153.asomxr (mgr.14195) 1218 : audit 0 from='client.15122 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:31 smithi153 bash[14340]: audit 2024-03-12T02:43:30.427983+0000 mon.smithi153 (mon.0) 1086 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:43:31.785 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:43:33.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:33 smithi163 bash[22347]: cluster 2024-03-12T02:43:31.787916+0000 mgr.smithi153.asomxr (mgr.14195) 1219 : cluster 0 pgmap v1016: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:33.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:33 smithi153 bash[14340]: cluster 2024-03-12T02:43:31.787916+0000 mgr.smithi153.asomxr (mgr.14195) 1219 : cluster 0 pgmap v1016: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:35.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:35 smithi163 bash[22347]: cluster 2024-03-12T02:43:33.788559+0000 mgr.smithi153.asomxr (mgr.14195) 1220 : cluster 0 pgmap v1017: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:35.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:35 smithi153 bash[14340]: cluster 2024-03-12T02:43:33.788559+0000 mgr.smithi153.asomxr (mgr.14195) 1220 : cluster 0 pgmap v1017: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:36.531 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:43:37.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:37 smithi163 bash[22347]: cluster 2024-03-12T02:43:35.789255+0000 mgr.smithi153.asomxr (mgr.14195) 1221 : cluster 0 pgmap v1018: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:37.510 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:37 smithi153 bash[14340]: cluster 2024-03-12T02:43:35.789255+0000 mgr.smithi153.asomxr (mgr.14195) 1221 : cluster 0 pgmap v1018: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:39.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:39 smithi163 bash[22347]: cluster 2024-03-12T02:43:37.789882+0000 mgr.smithi153.asomxr (mgr.14195) 1222 : cluster 0 pgmap v1019: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:39.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:39 smithi153 bash[14340]: cluster 2024-03-12T02:43:37.789882+0000 mgr.smithi153.asomxr (mgr.14195) 1222 : cluster 0 pgmap v1019: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:39.752 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:43:39.752 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:43:40.572 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:43:41.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:41 smithi163 bash[22347]: audit 2024-03-12T02:43:39.736619+0000 mgr.smithi153.asomxr (mgr.14195) 1223 : audit 0 from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:41.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:41 smithi163 bash[22347]: cluster 2024-03-12T02:43:39.790525+0000 mgr.smithi153.asomxr (mgr.14195) 1224 : cluster 0 pgmap v1020: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:41.573 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:43:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:41 smithi153 bash[14340]: audit 2024-03-12T02:43:39.736619+0000 mgr.smithi153.asomxr (mgr.14195) 1223 : audit 0 from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:41 smithi153 bash[14340]: cluster 2024-03-12T02:43:39.790525+0000 mgr.smithi153.asomxr (mgr.14195) 1224 : cluster 0 pgmap v1020: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:43.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:43 smithi153 bash[14340]: cluster 2024-03-12T02:43:41.791161+0000 mgr.smithi153.asomxr (mgr.14195) 1225 : cluster 0 pgmap v1021: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:43.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:43 smithi163 bash[22347]: cluster 2024-03-12T02:43:41.791161+0000 mgr.smithi153.asomxr (mgr.14195) 1225 : cluster 0 pgmap v1021: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:45.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:45 smithi163 bash[22347]: cluster 2024-03-12T02:43:43.791801+0000 mgr.smithi153.asomxr (mgr.14195) 1226 : cluster 0 pgmap v1022: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:45.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:45 smithi153 bash[14340]: cluster 2024-03-12T02:43:43.791801+0000 mgr.smithi153.asomxr (mgr.14195) 1226 : cluster 0 pgmap v1022: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:46.324 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:43:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:46 smithi163 bash[22347]: audit 2024-03-12T02:43:45.428622+0000 mon.smithi153 (mon.0) 1087 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:43:46.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:46 smithi153 bash[14340]: audit 2024-03-12T02:43:45.428622+0000 mon.smithi153 (mon.0) 1087 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:43:47.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:47 smithi163 bash[22347]: cluster 2024-03-12T02:43:45.792449+0000 mgr.smithi153.asomxr (mgr.14195) 1227 : cluster 0 pgmap v1023: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:47.519 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:47 smithi153 bash[14340]: cluster 2024-03-12T02:43:45.792449+0000 mgr.smithi153.asomxr (mgr.14195) 1227 : cluster 0 pgmap v1023: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:49.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:49 smithi163 bash[22347]: cluster 2024-03-12T02:43:47.792965+0000 mgr.smithi153.asomxr (mgr.14195) 1228 : cluster 0 pgmap v1024: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:49.572 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:43:49.573 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:43:49.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:49 smithi153 bash[14340]: cluster 2024-03-12T02:43:47.792965+0000 mgr.smithi153.asomxr (mgr.14195) 1228 : cluster 0 pgmap v1024: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:50.416 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:43:51.416 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:43:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:51 smithi163 bash[22347]: audit 2024-03-12T02:43:49.560041+0000 mgr.smithi153.asomxr (mgr.14195) 1229 : audit 0 from='client.15130 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:51 smithi163 bash[22347]: cluster 2024-03-12T02:43:49.793610+0000 mgr.smithi153.asomxr (mgr.14195) 1230 : cluster 0 pgmap v1025: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:51.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:51 smithi153 bash[14340]: audit 2024-03-12T02:43:49.560041+0000 mgr.smithi153.asomxr (mgr.14195) 1229 : audit 0 from='client.15130 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:43:51.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:51 smithi153 bash[14340]: cluster 2024-03-12T02:43:49.793610+0000 mgr.smithi153.asomxr (mgr.14195) 1230 : cluster 0 pgmap v1025: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:53.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:53 smithi153 bash[14340]: cluster 2024-03-12T02:43:51.794202+0000 mgr.smithi153.asomxr (mgr.14195) 1231 : cluster 0 pgmap v1026: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:53.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:53 smithi163 bash[22347]: cluster 2024-03-12T02:43:51.794202+0000 mgr.smithi153.asomxr (mgr.14195) 1231 : cluster 0 pgmap v1026: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:55.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:55 smithi163 bash[22347]: cluster 2024-03-12T02:43:53.794861+0000 mgr.smithi153.asomxr (mgr.14195) 1232 : cluster 0 pgmap v1027: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:55.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:55 smithi153 bash[14340]: cluster 2024-03-12T02:43:53.794861+0000 mgr.smithi153.asomxr (mgr.14195) 1232 : cluster 0 pgmap v1027: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:56.168 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:43:57.447 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:57 smithi153 bash[14340]: cluster 2024-03-12T02:43:55.795538+0000 mgr.smithi153.asomxr (mgr.14195) 1233 : cluster 0 pgmap v1028: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:57.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:57 smithi163 bash[22347]: cluster 2024-03-12T02:43:55.795538+0000 mgr.smithi153.asomxr (mgr.14195) 1233 : cluster 0 pgmap v1028: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:59.321 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:43:59.321 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:43:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:43:59 smithi163 bash[22347]: cluster 2024-03-12T02:43:57.796286+0000 mgr.smithi153.asomxr (mgr.14195) 1234 : cluster 0 pgmap v1029: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:43:59.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:43:59 smithi153 bash[14340]: cluster 2024-03-12T02:43:57.796286+0000 mgr.smithi153.asomxr (mgr.14195) 1234 : cluster 0 pgmap v1029: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:00.209 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:44:01.210 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:44:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:01 smithi163 bash[22347]: audit 2024-03-12T02:43:59.306445+0000 mgr.smithi153.asomxr (mgr.14195) 1235 : audit 0 from='client.15134 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:01.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:01 smithi163 bash[22347]: cluster 2024-03-12T02:43:59.796936+0000 mgr.smithi153.asomxr (mgr.14195) 1236 : cluster 0 pgmap v1030: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:01.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:01 smithi163 bash[22347]: audit 2024-03-12T02:44:00.428743+0000 mon.smithi153 (mon.0) 1088 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:44:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:01 smithi153 bash[14340]: audit 2024-03-12T02:43:59.306445+0000 mgr.smithi153.asomxr (mgr.14195) 1235 : audit 0 from='client.15134 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:01.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:01 smithi153 bash[14340]: cluster 2024-03-12T02:43:59.796936+0000 mgr.smithi153.asomxr (mgr.14195) 1236 : cluster 0 pgmap v1030: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:01.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:01 smithi153 bash[14340]: audit 2024-03-12T02:44:00.428743+0000 mon.smithi153 (mon.0) 1088 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:44:03.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:03 smithi163 bash[22347]: cluster 2024-03-12T02:44:01.797586+0000 mgr.smithi153.asomxr (mgr.14195) 1237 : cluster 0 pgmap v1031: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:03.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:03 smithi153 bash[14340]: cluster 2024-03-12T02:44:01.797586+0000 mgr.smithi153.asomxr (mgr.14195) 1237 : cluster 0 pgmap v1031: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:05.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:05 smithi163 bash[22347]: cluster 2024-03-12T02:44:03.798115+0000 mgr.smithi153.asomxr (mgr.14195) 1238 : cluster 0 pgmap v1032: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:05.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:05 smithi153 bash[14340]: cluster 2024-03-12T02:44:03.798115+0000 mgr.smithi153.asomxr (mgr.14195) 1238 : cluster 0 pgmap v1032: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:05.955 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:44:07.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:07 smithi153 bash[14340]: cluster 2024-03-12T02:44:05.798711+0000 mgr.smithi153.asomxr (mgr.14195) 1239 : cluster 0 pgmap v1033: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:07 smithi163 bash[22347]: cluster 2024-03-12T02:44:05.798711+0000 mgr.smithi153.asomxr (mgr.14195) 1239 : cluster 0 pgmap v1033: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:09.110 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:44:09.111 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:44:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:09 smithi163 bash[22347]: cluster 2024-03-12T02:44:07.799312+0000 mgr.smithi153.asomxr (mgr.14195) 1240 : cluster 0 pgmap v1034: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:09.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:09 smithi153 bash[14340]: cluster 2024-03-12T02:44:07.799312+0000 mgr.smithi153.asomxr (mgr.14195) 1240 : cluster 0 pgmap v1034: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:09.880 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:44:10.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:10 smithi163 bash[22347]: audit 2024-03-12T02:44:09.098518+0000 mgr.smithi153.asomxr (mgr.14195) 1241 : audit 0 from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:10.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:10 smithi153 bash[14340]: audit 2024-03-12T02:44:09.098518+0000 mgr.smithi153.asomxr (mgr.14195) 1241 : audit 0 from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:10.882 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:44:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:11 smithi163 bash[22347]: cluster 2024-03-12T02:44:09.799913+0000 mgr.smithi153.asomxr (mgr.14195) 1242 : cluster 0 pgmap v1035: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:11.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:11 smithi153 bash[14340]: cluster 2024-03-12T02:44:09.799913+0000 mgr.smithi153.asomxr (mgr.14195) 1242 : cluster 0 pgmap v1035: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:13.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:13 smithi153 bash[14340]: cluster 2024-03-12T02:44:11.800528+0000 mgr.smithi153.asomxr (mgr.14195) 1243 : cluster 0 pgmap v1036: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:13.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:13 smithi163 bash[22347]: cluster 2024-03-12T02:44:11.800528+0000 mgr.smithi153.asomxr (mgr.14195) 1243 : cluster 0 pgmap v1036: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:15.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:15 smithi163 bash[22347]: cluster 2024-03-12T02:44:13.801127+0000 mgr.smithi153.asomxr (mgr.14195) 1244 : cluster 0 pgmap v1037: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:15.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:15 smithi153 bash[14340]: cluster 2024-03-12T02:44:13.801127+0000 mgr.smithi153.asomxr (mgr.14195) 1244 : cluster 0 pgmap v1037: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:15.633 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:44:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:16 smithi163 bash[22347]: audit 2024-03-12T02:44:15.429204+0000 mon.smithi153 (mon.0) 1089 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:44:16.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:16 smithi153 bash[14340]: audit 2024-03-12T02:44:15.429204+0000 mon.smithi153 (mon.0) 1089 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:44:17.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:17 smithi153 bash[14340]: cluster 2024-03-12T02:44:15.801622+0000 mgr.smithi153.asomxr (mgr.14195) 1245 : cluster 0 pgmap v1038: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:17.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:17 smithi163 bash[22347]: cluster 2024-03-12T02:44:15.801622+0000 mgr.smithi153.asomxr (mgr.14195) 1245 : cluster 0 pgmap v1038: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:18.983 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:44:18.983 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:44:19.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:19 smithi153 bash[14340]: cluster 2024-03-12T02:44:17.802189+0000 mgr.smithi153.asomxr (mgr.14195) 1246 : cluster 0 pgmap v1039: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:19 smithi163 bash[22347]: cluster 2024-03-12T02:44:17.802189+0000 mgr.smithi153.asomxr (mgr.14195) 1246 : cluster 0 pgmap v1039: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:19.752 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:44:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:20 smithi163 bash[22347]: audit 2024-03-12T02:44:18.968958+0000 mgr.smithi153.asomxr (mgr.14195) 1247 : audit 0 from='client.15142 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:20.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:20 smithi153 bash[14340]: audit 2024-03-12T02:44:18.968958+0000 mgr.smithi153.asomxr (mgr.14195) 1247 : audit 0 from='client.15142 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:20.753 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:44:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:21 smithi163 bash[22347]: cluster 2024-03-12T02:44:19.802861+0000 mgr.smithi153.asomxr (mgr.14195) 1248 : cluster 0 pgmap v1040: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:21.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:21 smithi153 bash[14340]: cluster 2024-03-12T02:44:19.802861+0000 mgr.smithi153.asomxr (mgr.14195) 1248 : cluster 0 pgmap v1040: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:23.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:23 smithi153 bash[14340]: cluster 2024-03-12T02:44:21.803614+0000 mgr.smithi153.asomxr (mgr.14195) 1249 : cluster 0 pgmap v1041: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:23.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:23 smithi163 bash[22347]: cluster 2024-03-12T02:44:21.803614+0000 mgr.smithi153.asomxr (mgr.14195) 1249 : cluster 0 pgmap v1041: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:25.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:25 smithi153 bash[14340]: cluster 2024-03-12T02:44:23.804267+0000 mgr.smithi153.asomxr (mgr.14195) 1250 : cluster 0 pgmap v1042: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:25.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:25 smithi163 bash[22347]: cluster 2024-03-12T02:44:23.804267+0000 mgr.smithi153.asomxr (mgr.14195) 1250 : cluster 0 pgmap v1042: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:25.507 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:44:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:26 smithi163 bash[22347]: audit 2024-03-12T02:44:25.894478+0000 mon.smithi153 (mon.0) 1090 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:44:26.510 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:26 smithi153 bash[14340]: audit 2024-03-12T02:44:25.894478+0000 mon.smithi153 (mon.0) 1090 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:44:27.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:27 smithi163 bash[22347]: cluster 2024-03-12T02:44:25.804883+0000 mgr.smithi153.asomxr (mgr.14195) 1251 : cluster 0 pgmap v1043: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:27.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:27 smithi163 bash[22347]: audit 2024-03-12T02:44:26.422917+0000 mon.smithi153 (mon.0) 1091 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:44:27.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:27 smithi163 bash[22347]: audit 2024-03-12T02:44:26.424344+0000 mon.smithi153 (mon.0) 1092 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:44:27.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:27 smithi163 bash[22347]: audit 2024-03-12T02:44:26.431384+0000 mon.smithi153 (mon.0) 1093 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:44:27.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:27 smithi163 bash[22347]: audit 2024-03-12T02:44:26.435962+0000 mon.smithi153 (mon.0) 1094 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:44:27.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:27 smithi153 bash[14340]: cluster 2024-03-12T02:44:25.804883+0000 mgr.smithi153.asomxr (mgr.14195) 1251 : cluster 0 pgmap v1043: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:27.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:27 smithi153 bash[14340]: audit 2024-03-12T02:44:26.422917+0000 mon.smithi153 (mon.0) 1091 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:44:27.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:27 smithi153 bash[14340]: audit 2024-03-12T02:44:26.424344+0000 mon.smithi153 (mon.0) 1092 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:44:27.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:27 smithi153 bash[14340]: audit 2024-03-12T02:44:26.431384+0000 mon.smithi153 (mon.0) 1093 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:44:27.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:27 smithi153 bash[14340]: audit 2024-03-12T02:44:26.435962+0000 mon.smithi153 (mon.0) 1094 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:44:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:28 smithi163 bash[22347]: cluster 2024-03-12T02:44:26.426875+0000 mgr.smithi153.asomxr (mgr.14195) 1252 : cluster 0 pgmap v1044: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:28.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:28 smithi153 bash[14340]: cluster 2024-03-12T02:44:26.426875+0000 mgr.smithi153.asomxr (mgr.14195) 1252 : cluster 0 pgmap v1044: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:28.664 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:44:28.664 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:44:29.418 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:44:30.420 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:44:30.433 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:30 smithi153 bash[14340]: cluster 2024-03-12T02:44:28.427340+0000 mgr.smithi153.asomxr (mgr.14195) 1253 : cluster 0 pgmap v1045: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:30.433 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:30 smithi153 bash[14340]: audit 2024-03-12T02:44:28.647487+0000 mgr.smithi153.asomxr (mgr.14195) 1254 : audit 0 from='client.15146 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:30.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:30 smithi163 bash[22347]: cluster 2024-03-12T02:44:28.427340+0000 mgr.smithi153.asomxr (mgr.14195) 1253 : cluster 0 pgmap v1045: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:30.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:30 smithi163 bash[22347]: audit 2024-03-12T02:44:28.647487+0000 mgr.smithi153.asomxr (mgr.14195) 1254 : audit 0 from='client.15146 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:31 smithi163 bash[22347]: audit 2024-03-12T02:44:30.429687+0000 mon.smithi153 (mon.0) 1095 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:44:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:31 smithi153 bash[14340]: audit 2024-03-12T02:44:30.429687+0000 mon.smithi153 (mon.0) 1095 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:44:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:32 smithi163 bash[22347]: cluster 2024-03-12T02:44:30.428052+0000 mgr.smithi153.asomxr (mgr.14195) 1255 : cluster 0 pgmap v1046: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:32.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:32 smithi153 bash[14340]: cluster 2024-03-12T02:44:30.428052+0000 mgr.smithi153.asomxr (mgr.14195) 1255 : cluster 0 pgmap v1046: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:34.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:34 smithi163 bash[22347]: cluster 2024-03-12T02:44:32.428682+0000 mgr.smithi153.asomxr (mgr.14195) 1256 : cluster 0 pgmap v1047: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:34.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:34 smithi153 bash[14340]: cluster 2024-03-12T02:44:32.428682+0000 mgr.smithi153.asomxr (mgr.14195) 1256 : cluster 0 pgmap v1047: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:35.169 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:44:35.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:35 smithi153 bash[14340]: cluster 2024-03-12T02:44:34.429358+0000 mgr.smithi153.asomxr (mgr.14195) 1257 : cluster 0 pgmap v1048: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:35.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:35 smithi163 bash[22347]: cluster 2024-03-12T02:44:34.429358+0000 mgr.smithi153.asomxr (mgr.14195) 1257 : cluster 0 pgmap v1048: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:37.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:37 smithi153 bash[14340]: cluster 2024-03-12T02:44:36.429918+0000 mgr.smithi153.asomxr (mgr.14195) 1258 : cluster 0 pgmap v1049: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:37.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:37 smithi163 bash[22347]: cluster 2024-03-12T02:44:36.429918+0000 mgr.smithi153.asomxr (mgr.14195) 1258 : cluster 0 pgmap v1049: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:38.479 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:44:38.479 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:44:39.225 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:44:40.226 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:44:40.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:40 smithi163 bash[22347]: cluster 2024-03-12T02:44:38.430618+0000 mgr.smithi153.asomxr (mgr.14195) 1259 : cluster 0 pgmap v1050: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:40.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:40 smithi163 bash[22347]: audit 2024-03-12T02:44:38.463170+0000 mgr.smithi153.asomxr (mgr.14195) 1260 : audit 0 from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:40.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:40 smithi153 bash[14340]: cluster 2024-03-12T02:44:38.430618+0000 mgr.smithi153.asomxr (mgr.14195) 1259 : cluster 0 pgmap v1050: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:40.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:40 smithi153 bash[14340]: audit 2024-03-12T02:44:38.463170+0000 mgr.smithi153.asomxr (mgr.14195) 1260 : audit 0 from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:42.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:42 smithi163 bash[22347]: cluster 2024-03-12T02:44:40.431240+0000 mgr.smithi153.asomxr (mgr.14195) 1261 : cluster 0 pgmap v1051: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:42.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:42 smithi153 bash[14340]: cluster 2024-03-12T02:44:40.431240+0000 mgr.smithi153.asomxr (mgr.14195) 1261 : cluster 0 pgmap v1051: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:44.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:44 smithi163 bash[22347]: cluster 2024-03-12T02:44:42.431862+0000 mgr.smithi153.asomxr (mgr.14195) 1262 : cluster 0 pgmap v1052: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:44.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:44 smithi153 bash[14340]: cluster 2024-03-12T02:44:42.431862+0000 mgr.smithi153.asomxr (mgr.14195) 1262 : cluster 0 pgmap v1052: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:44.975 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:44:46.314 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:46 smithi153 bash[14340]: cluster 2024-03-12T02:44:44.432553+0000 mgr.smithi153.asomxr (mgr.14195) 1263 : cluster 0 pgmap v1053: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:46.314 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:46 smithi153 bash[14340]: audit 2024-03-12T02:44:45.429381+0000 mon.smithi153 (mon.0) 1096 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:44:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:46 smithi163 bash[22347]: cluster 2024-03-12T02:44:44.432553+0000 mgr.smithi153.asomxr (mgr.14195) 1263 : cluster 0 pgmap v1053: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:46 smithi163 bash[22347]: audit 2024-03-12T02:44:45.429381+0000 mon.smithi153 (mon.0) 1096 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:44:48.271 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:44:48.272 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:44:48.439 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:48 smithi153 bash[14340]: cluster 2024-03-12T02:44:46.433245+0000 mgr.smithi153.asomxr (mgr.14195) 1264 : cluster 0 pgmap v1054: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:48.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:48 smithi163 bash[22347]: cluster 2024-03-12T02:44:46.433245+0000 mgr.smithi153.asomxr (mgr.14195) 1264 : cluster 0 pgmap v1054: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:49.012 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:44:50.013 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:44:50.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:50 smithi153 bash[14340]: audit 2024-03-12T02:44:48.257015+0000 mgr.smithi153.asomxr (mgr.14195) 1265 : audit 0 from='client.15154 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:50.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:50 smithi153 bash[14340]: cluster 2024-03-12T02:44:48.433938+0000 mgr.smithi153.asomxr (mgr.14195) 1266 : cluster 0 pgmap v1055: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:50 smithi163 bash[22347]: audit 2024-03-12T02:44:48.257015+0000 mgr.smithi153.asomxr (mgr.14195) 1265 : audit 0 from='client.15154 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:50 smithi163 bash[22347]: cluster 2024-03-12T02:44:48.433938+0000 mgr.smithi153.asomxr (mgr.14195) 1266 : cluster 0 pgmap v1055: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:52 smithi163 bash[22347]: cluster 2024-03-12T02:44:50.434694+0000 mgr.smithi153.asomxr (mgr.14195) 1267 : cluster 0 pgmap v1056: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:52.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:52 smithi153 bash[14340]: cluster 2024-03-12T02:44:50.434694+0000 mgr.smithi153.asomxr (mgr.14195) 1267 : cluster 0 pgmap v1056: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:54 smithi163 bash[22347]: cluster 2024-03-12T02:44:52.435449+0000 mgr.smithi153.asomxr (mgr.14195) 1268 : cluster 0 pgmap v1057: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:54.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:54 smithi153 bash[14340]: cluster 2024-03-12T02:44:52.435449+0000 mgr.smithi153.asomxr (mgr.14195) 1268 : cluster 0 pgmap v1057: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:54.765 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:44:56.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:56 smithi153 bash[14340]: cluster 2024-03-12T02:44:54.436143+0000 mgr.smithi153.asomxr (mgr.14195) 1269 : cluster 0 pgmap v1058: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:56.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:56 smithi163 bash[22347]: cluster 2024-03-12T02:44:54.436143+0000 mgr.smithi153.asomxr (mgr.14195) 1269 : cluster 0 pgmap v1058: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:58.133 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:44:58.134 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:44:58.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:58 smithi163 bash[22347]: cluster 2024-03-12T02:44:56.436745+0000 mgr.smithi153.asomxr (mgr.14195) 1270 : cluster 0 pgmap v1059: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:58.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:58 smithi153 bash[14340]: cluster 2024-03-12T02:44:56.436745+0000 mgr.smithi153.asomxr (mgr.14195) 1270 : cluster 0 pgmap v1059: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:44:58.933 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:44:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:44:59 smithi163 bash[22347]: audit 2024-03-12T02:44:58.118278+0000 mgr.smithi153.asomxr (mgr.14195) 1271 : audit 0 from='client.15158 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:59.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:44:59 smithi153 bash[14340]: audit 2024-03-12T02:44:58.118278+0000 mgr.smithi153.asomxr (mgr.14195) 1271 : audit 0 from='client.15158 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:44:59.934 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:45:00.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:00 smithi163 bash[22347]: cluster 2024-03-12T02:44:58.437222+0000 mgr.smithi153.asomxr (mgr.14195) 1272 : cluster 0 pgmap v1060: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:00.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:00 smithi153 bash[14340]: cluster 2024-03-12T02:44:58.437222+0000 mgr.smithi153.asomxr (mgr.14195) 1272 : cluster 0 pgmap v1060: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:01.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:01 smithi153 bash[14340]: audit 2024-03-12T02:45:00.430414+0000 mon.smithi153 (mon.0) 1097 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:45:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:01 smithi163 bash[22347]: audit 2024-03-12T02:45:00.430414+0000 mon.smithi153 (mon.0) 1097 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:45:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:02 smithi163 bash[22347]: cluster 2024-03-12T02:45:00.439039+0000 mgr.smithi153.asomxr (mgr.14195) 1273 : cluster 0 pgmap v1061: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:02.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:02 smithi153 bash[14340]: cluster 2024-03-12T02:45:00.439039+0000 mgr.smithi153.asomxr (mgr.14195) 1273 : cluster 0 pgmap v1061: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:04 smithi163 bash[22347]: cluster 2024-03-12T02:45:02.439648+0000 mgr.smithi153.asomxr (mgr.14195) 1274 : cluster 0 pgmap v1062: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:04.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:04 smithi153 bash[14340]: cluster 2024-03-12T02:45:02.439648+0000 mgr.smithi153.asomxr (mgr.14195) 1274 : cluster 0 pgmap v1062: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:04.680 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:45:06.486 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:06 smithi163 bash[22347]: cluster 2024-03-12T02:45:04.440309+0000 mgr.smithi153.asomxr (mgr.14195) 1275 : cluster 0 pgmap v1063: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:06 smithi153 bash[14340]: cluster 2024-03-12T02:45:04.440309+0000 mgr.smithi153.asomxr (mgr.14195) 1275 : cluster 0 pgmap v1063: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:08.140 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:45:08.140 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:45:08.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:08 smithi153 bash[14340]: cluster 2024-03-12T02:45:06.440978+0000 mgr.smithi153.asomxr (mgr.14195) 1276 : cluster 0 pgmap v1064: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:08.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:08 smithi163 bash[22347]: cluster 2024-03-12T02:45:06.440978+0000 mgr.smithi153.asomxr (mgr.14195) 1276 : cluster 0 pgmap v1064: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:08.946 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:45:09.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:09 smithi153 bash[14340]: audit 2024-03-12T02:45:08.123519+0000 mgr.smithi153.asomxr (mgr.14195) 1277 : audit 0 from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:09.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:09 smithi163 bash[22347]: audit 2024-03-12T02:45:08.123519+0000 mgr.smithi153.asomxr (mgr.14195) 1277 : audit 0 from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:09.947 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:45:10.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:10 smithi153 bash[14340]: cluster 2024-03-12T02:45:08.441513+0000 mgr.smithi153.asomxr (mgr.14195) 1278 : cluster 0 pgmap v1065: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:10.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:10 smithi163 bash[22347]: cluster 2024-03-12T02:45:08.441513+0000 mgr.smithi153.asomxr (mgr.14195) 1278 : cluster 0 pgmap v1065: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:11.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:11 smithi153 bash[14340]: cluster 2024-03-12T02:45:10.442143+0000 mgr.smithi153.asomxr (mgr.14195) 1279 : cluster 0 pgmap v1066: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:11.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:11 smithi163 bash[22347]: cluster 2024-03-12T02:45:10.442143+0000 mgr.smithi153.asomxr (mgr.14195) 1279 : cluster 0 pgmap v1066: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:14 smithi163 bash[22347]: cluster 2024-03-12T02:45:12.442775+0000 mgr.smithi153.asomxr (mgr.14195) 1280 : cluster 0 pgmap v1067: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:14.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:14 smithi153 bash[14340]: cluster 2024-03-12T02:45:12.442775+0000 mgr.smithi153.asomxr (mgr.14195) 1280 : cluster 0 pgmap v1067: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:14.706 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:45:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:16 smithi153 bash[14340]: cluster 2024-03-12T02:45:14.443479+0000 mgr.smithi153.asomxr (mgr.14195) 1281 : cluster 0 pgmap v1068: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:16 smithi153 bash[14340]: audit 2024-03-12T02:45:15.430714+0000 mon.smithi153 (mon.0) 1098 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:45:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:16 smithi163 bash[22347]: cluster 2024-03-12T02:45:14.443479+0000 mgr.smithi153.asomxr (mgr.14195) 1281 : cluster 0 pgmap v1068: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:16 smithi163 bash[22347]: audit 2024-03-12T02:45:15.430714+0000 mon.smithi153 (mon.0) 1098 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:45:18.128 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:45:18.128 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:45:18.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:18 smithi163 bash[22347]: cluster 2024-03-12T02:45:16.444096+0000 mgr.smithi153.asomxr (mgr.14195) 1282 : cluster 0 pgmap v1069: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:18.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:18 smithi153 bash[14340]: cluster 2024-03-12T02:45:16.444096+0000 mgr.smithi153.asomxr (mgr.14195) 1282 : cluster 0 pgmap v1069: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:18.866 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:45:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:19 smithi163 bash[22347]: audit 2024-03-12T02:45:18.112387+0000 mgr.smithi153.asomxr (mgr.14195) 1283 : audit 0 from='client.15166 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:19.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:19 smithi153 bash[14340]: audit 2024-03-12T02:45:18.112387+0000 mgr.smithi153.asomxr (mgr.14195) 1283 : audit 0 from='client.15166 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:19.867 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:45:20.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:20 smithi153 bash[14340]: cluster 2024-03-12T02:45:18.444776+0000 mgr.smithi153.asomxr (mgr.14195) 1284 : cluster 0 pgmap v1070: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:20 smithi163 bash[22347]: cluster 2024-03-12T02:45:18.444776+0000 mgr.smithi153.asomxr (mgr.14195) 1284 : cluster 0 pgmap v1070: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:22.487 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:22 smithi163 bash[22347]: cluster 2024-03-12T02:45:20.445474+0000 mgr.smithi153.asomxr (mgr.14195) 1285 : cluster 0 pgmap v1071: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:22.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:22 smithi153 bash[14340]: cluster 2024-03-12T02:45:20.445474+0000 mgr.smithi153.asomxr (mgr.14195) 1285 : cluster 0 pgmap v1071: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:24 smithi163 bash[22347]: cluster 2024-03-12T02:45:22.446066+0000 mgr.smithi153.asomxr (mgr.14195) 1286 : cluster 0 pgmap v1072: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:24.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:24 smithi153 bash[14340]: cluster 2024-03-12T02:45:22.446066+0000 mgr.smithi153.asomxr (mgr.14195) 1286 : cluster 0 pgmap v1072: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:24.788 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:45:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:26 smithi163 bash[22347]: cluster 2024-03-12T02:45:24.446741+0000 mgr.smithi153.asomxr (mgr.14195) 1287 : cluster 0 pgmap v1073: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:26.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:26 smithi153 bash[14340]: cluster 2024-03-12T02:45:24.446741+0000 mgr.smithi153.asomxr (mgr.14195) 1287 : cluster 0 pgmap v1073: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:27.476 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:27 smithi153 bash[14340]: audit 2024-03-12T02:45:26.522390+0000 mon.smithi153 (mon.0) 1099 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:45:27.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:27 smithi163 bash[22347]: audit 2024-03-12T02:45:26.522390+0000 mon.smithi153 (mon.0) 1099 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:45:27.903 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:45:27.903 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:45:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:28 smithi163 bash[22347]: cluster 2024-03-12T02:45:26.447361+0000 mgr.smithi153.asomxr (mgr.14195) 1288 : cluster 0 pgmap v1074: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:28.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:28 smithi153 bash[14340]: cluster 2024-03-12T02:45:26.447361+0000 mgr.smithi153.asomxr (mgr.14195) 1288 : cluster 0 pgmap v1074: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:28.739 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:45:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:29 smithi163 bash[22347]: audit 2024-03-12T02:45:27.887315+0000 mgr.smithi153.asomxr (mgr.14195) 1289 : audit 0 from='client.15170 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:29.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:29 smithi153 bash[14340]: audit 2024-03-12T02:45:27.887315+0000 mgr.smithi153.asomxr (mgr.14195) 1289 : audit 0 from='client.15170 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:29.740 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:45:30.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:30 smithi163 bash[22347]: cluster 2024-03-12T02:45:28.448014+0000 mgr.smithi153.asomxr (mgr.14195) 1290 : cluster 0 pgmap v1075: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:30.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:30 smithi153 bash[14340]: cluster 2024-03-12T02:45:28.448014+0000 mgr.smithi153.asomxr (mgr.14195) 1290 : cluster 0 pgmap v1075: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:31.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:31 smithi163 bash[22347]: audit 2024-03-12T02:45:30.431208+0000 mon.smithi153 (mon.0) 1100 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:45:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:31 smithi153 bash[14340]: audit 2024-03-12T02:45:30.431208+0000 mon.smithi153 (mon.0) 1100 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:45:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:32 smithi163 bash[22347]: cluster 2024-03-12T02:45:30.448550+0000 mgr.smithi153.asomxr (mgr.14195) 1291 : cluster 0 pgmap v1076: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:32.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:32 smithi153 bash[14340]: cluster 2024-03-12T02:45:30.448550+0000 mgr.smithi153.asomxr (mgr.14195) 1291 : cluster 0 pgmap v1076: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:34.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:34 smithi153 bash[14340]: cluster 2024-03-12T02:45:32.449152+0000 mgr.smithi153.asomxr (mgr.14195) 1292 : cluster 0 pgmap v1077: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:34.669 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:34 smithi163 bash[22347]: cluster 2024-03-12T02:45:32.449152+0000 mgr.smithi153.asomxr (mgr.14195) 1292 : cluster 0 pgmap v1077: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:34.688 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:45:36.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:36 smithi153 bash[14340]: cluster 2024-03-12T02:45:34.449823+0000 mgr.smithi153.asomxr (mgr.14195) 1293 : cluster 0 pgmap v1078: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:36.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:36 smithi163 bash[22347]: cluster 2024-03-12T02:45:34.449823+0000 mgr.smithi153.asomxr (mgr.14195) 1293 : cluster 0 pgmap v1078: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:38.141 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:45:38.142 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:45:38.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:38 smithi163 bash[22347]: cluster 2024-03-12T02:45:36.450280+0000 mgr.smithi153.asomxr (mgr.14195) 1294 : cluster 0 pgmap v1079: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:38.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:38 smithi153 bash[14340]: cluster 2024-03-12T02:45:36.450280+0000 mgr.smithi153.asomxr (mgr.14195) 1294 : cluster 0 pgmap v1079: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:38.918 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:45:39.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:39 smithi153 bash[14340]: audit 2024-03-12T02:45:38.129619+0000 mgr.smithi153.asomxr (mgr.14195) 1295 : audit 0 from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:39.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:39 smithi153 bash[14340]: audit 2024-03-12T02:45:38.696607+0000 mon.smithi153 (mon.0) 1101 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:45:39.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:39 smithi153 bash[14340]: audit 2024-03-12T02:45:38.703428+0000 mon.smithi153 (mon.0) 1102 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:45:39.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:39 smithi163 bash[22347]: audit 2024-03-12T02:45:38.129619+0000 mgr.smithi153.asomxr (mgr.14195) 1295 : audit 0 from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:39.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:39 smithi163 bash[22347]: audit 2024-03-12T02:45:38.696607+0000 mon.smithi153 (mon.0) 1101 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:45:39.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:39 smithi163 bash[22347]: audit 2024-03-12T02:45:38.703428+0000 mon.smithi153 (mon.0) 1102 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:45:39.920 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:45:40.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:40 smithi153 bash[14340]: cluster 2024-03-12T02:45:38.450835+0000 mgr.smithi153.asomxr (mgr.14195) 1296 : cluster 0 pgmap v1080: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:40.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:40 smithi153 bash[14340]: audit 2024-03-12T02:45:39.806956+0000 mon.smithi153 (mon.0) 1103 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:45:40.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:40 smithi153 bash[14340]: audit 2024-03-12T02:45:39.816763+0000 mon.smithi153 (mon.0) 1104 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:45:40.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:40 smithi153 bash[14340]: audit 2024-03-12T02:45:39.818849+0000 mon.smithi153 (mon.0) 1105 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:45:40.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:40 smithi153 bash[14340]: audit 2024-03-12T02:45:39.820270+0000 mon.smithi153 (mon.0) 1106 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:45:40.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:40 smithi153 bash[14340]: audit 2024-03-12T02:45:39.829107+0000 mon.smithi153 (mon.0) 1107 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:45:40.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:40 smithi153 bash[14340]: audit 2024-03-12T02:45:39.834420+0000 mon.smithi153 (mon.0) 1108 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:45:40.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:40 smithi163 bash[22347]: cluster 2024-03-12T02:45:38.450835+0000 mgr.smithi153.asomxr (mgr.14195) 1296 : cluster 0 pgmap v1080: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:40.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:40 smithi163 bash[22347]: audit 2024-03-12T02:45:39.806956+0000 mon.smithi153 (mon.0) 1103 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:45:40.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:40 smithi163 bash[22347]: audit 2024-03-12T02:45:39.816763+0000 mon.smithi153 (mon.0) 1104 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:45:40.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:40 smithi163 bash[22347]: audit 2024-03-12T02:45:39.818849+0000 mon.smithi153 (mon.0) 1105 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:45:40.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:40 smithi163 bash[22347]: audit 2024-03-12T02:45:39.820270+0000 mon.smithi153 (mon.0) 1106 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:45:40.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:40 smithi163 bash[22347]: audit 2024-03-12T02:45:39.829107+0000 mon.smithi153 (mon.0) 1107 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:45:40.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:40 smithi163 bash[22347]: audit 2024-03-12T02:45:39.834420+0000 mon.smithi153 (mon.0) 1108 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:45:41.516 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:41 smithi153 bash[14340]: cluster 2024-03-12T02:45:39.822930+0000 mgr.smithi153.asomxr (mgr.14195) 1297 : cluster 0 pgmap v1081: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:41.517 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:41 smithi153 bash[14340]: cluster 2024-03-12T02:45:39.823261+0000 mgr.smithi153.asomxr (mgr.14195) 1298 : cluster 0 pgmap v1082: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:41.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:41 smithi163 bash[22347]: cluster 2024-03-12T02:45:39.822930+0000 mgr.smithi153.asomxr (mgr.14195) 1297 : cluster 0 pgmap v1081: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:41.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:41 smithi163 bash[22347]: cluster 2024-03-12T02:45:39.823261+0000 mgr.smithi153.asomxr (mgr.14195) 1298 : cluster 0 pgmap v1082: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:42.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:42 smithi153 bash[14340]: cluster 2024-03-12T02:45:41.823839+0000 mgr.smithi153.asomxr (mgr.14195) 1299 : cluster 0 pgmap v1083: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:42 smithi163 bash[22347]: cluster 2024-03-12T02:45:41.823839+0000 mgr.smithi153.asomxr (mgr.14195) 1299 : cluster 0 pgmap v1083: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:44.677 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:45:45.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:45 smithi163 bash[22347]: cluster 2024-03-12T02:45:43.824499+0000 mgr.smithi153.asomxr (mgr.14195) 1300 : cluster 0 pgmap v1084: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:45.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:45 smithi153 bash[14340]: cluster 2024-03-12T02:45:43.824499+0000 mgr.smithi153.asomxr (mgr.14195) 1300 : cluster 0 pgmap v1084: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:46 smithi153 bash[14340]: audit 2024-03-12T02:45:45.431979+0000 mon.smithi153 (mon.0) 1109 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:45:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:46 smithi163 bash[22347]: audit 2024-03-12T02:45:45.431979+0000 mon.smithi153 (mon.0) 1109 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:45:47.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:47 smithi163 bash[22347]: cluster 2024-03-12T02:45:45.824943+0000 mgr.smithi153.asomxr (mgr.14195) 1301 : cluster 0 pgmap v1085: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:47.571 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:47 smithi153 bash[14340]: cluster 2024-03-12T02:45:45.824943+0000 mgr.smithi153.asomxr (mgr.14195) 1301 : cluster 0 pgmap v1085: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:48.042 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:45:48.042 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:45:48.836 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:45:49.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:49 smithi163 bash[22347]: cluster 2024-03-12T02:45:47.825497+0000 mgr.smithi153.asomxr (mgr.14195) 1302 : cluster 0 pgmap v1086: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:49.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:49 smithi163 bash[22347]: audit 2024-03-12T02:45:48.026008+0000 mgr.smithi153.asomxr (mgr.14195) 1303 : audit 0 from='client.15178 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:49.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:49 smithi153 bash[14340]: cluster 2024-03-12T02:45:47.825497+0000 mgr.smithi153.asomxr (mgr.14195) 1302 : cluster 0 pgmap v1086: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:49.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:49 smithi153 bash[14340]: audit 2024-03-12T02:45:48.026008+0000 mgr.smithi153.asomxr (mgr.14195) 1303 : audit 0 from='client.15178 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:49.837 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:45:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:51 smithi163 bash[22347]: cluster 2024-03-12T02:45:49.826236+0000 mgr.smithi153.asomxr (mgr.14195) 1304 : cluster 0 pgmap v1087: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:51.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:51 smithi153 bash[14340]: cluster 2024-03-12T02:45:49.826236+0000 mgr.smithi153.asomxr (mgr.14195) 1304 : cluster 0 pgmap v1087: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:53.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:53 smithi153 bash[14340]: cluster 2024-03-12T02:45:51.826969+0000 mgr.smithi153.asomxr (mgr.14195) 1305 : cluster 0 pgmap v1088: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:53.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:53 smithi163 bash[22347]: cluster 2024-03-12T02:45:51.826969+0000 mgr.smithi153.asomxr (mgr.14195) 1305 : cluster 0 pgmap v1088: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:54.590 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:45:55.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:55 smithi163 bash[22347]: cluster 2024-03-12T02:45:53.827599+0000 mgr.smithi153.asomxr (mgr.14195) 1306 : cluster 0 pgmap v1089: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:55.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:55 smithi153 bash[14340]: cluster 2024-03-12T02:45:53.827599+0000 mgr.smithi153.asomxr (mgr.14195) 1306 : cluster 0 pgmap v1089: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:57.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:57 smithi163 bash[22347]: cluster 2024-03-12T02:45:55.828276+0000 mgr.smithi153.asomxr (mgr.14195) 1307 : cluster 0 pgmap v1090: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:57.513 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:57 smithi153 bash[14340]: cluster 2024-03-12T02:45:55.828276+0000 mgr.smithi153.asomxr (mgr.14195) 1307 : cluster 0 pgmap v1090: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:57.974 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:45:57.974 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:45:58.768 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:45:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:59 smithi163 bash[22347]: cluster 2024-03-12T02:45:57.828750+0000 mgr.smithi153.asomxr (mgr.14195) 1308 : cluster 0 pgmap v1091: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:45:59 smithi163 bash[22347]: audit 2024-03-12T02:45:57.958726+0000 mgr.smithi153.asomxr (mgr.14195) 1309 : audit 0 from='client.15182 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:59.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:59 smithi153 bash[14340]: cluster 2024-03-12T02:45:57.828750+0000 mgr.smithi153.asomxr (mgr.14195) 1308 : cluster 0 pgmap v1091: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:45:59.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:45:59 smithi153 bash[14340]: audit 2024-03-12T02:45:57.958726+0000 mgr.smithi153.asomxr (mgr.14195) 1309 : audit 0 from='client.15182 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:45:59.769 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:46:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:01 smithi163 bash[22347]: cluster 2024-03-12T02:45:59.829381+0000 mgr.smithi153.asomxr (mgr.14195) 1310 : cluster 0 pgmap v1092: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:01 smithi163 bash[22347]: audit 2024-03-12T02:46:00.432019+0000 mon.smithi153 (mon.0) 1110 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:46:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:01 smithi153 bash[14340]: cluster 2024-03-12T02:45:59.829381+0000 mgr.smithi153.asomxr (mgr.14195) 1310 : cluster 0 pgmap v1092: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:01.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:01 smithi153 bash[14340]: audit 2024-03-12T02:46:00.432019+0000 mon.smithi153 (mon.0) 1110 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:46:03.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:03 smithi163 bash[22347]: cluster 2024-03-12T02:46:01.829946+0000 mgr.smithi153.asomxr (mgr.14195) 1311 : cluster 0 pgmap v1093: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:03.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:03 smithi153 bash[14340]: cluster 2024-03-12T02:46:01.829946+0000 mgr.smithi153.asomxr (mgr.14195) 1311 : cluster 0 pgmap v1093: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:04.516 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:46:05.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:05 smithi153 bash[14340]: cluster 2024-03-12T02:46:03.830554+0000 mgr.smithi153.asomxr (mgr.14195) 1312 : cluster 0 pgmap v1094: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:05.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:05 smithi163 bash[22347]: cluster 2024-03-12T02:46:03.830554+0000 mgr.smithi153.asomxr (mgr.14195) 1312 : cluster 0 pgmap v1094: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:07.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:07 smithi153 bash[14340]: cluster 2024-03-12T02:46:05.831198+0000 mgr.smithi153.asomxr (mgr.14195) 1313 : cluster 0 pgmap v1095: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:07.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:07 smithi163 bash[22347]: cluster 2024-03-12T02:46:05.831198+0000 mgr.smithi153.asomxr (mgr.14195) 1313 : cluster 0 pgmap v1095: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:07.791 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:46:07.791 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:46:08.656 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:46:09.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:09 smithi153 bash[14340]: audit 2024-03-12T02:46:07.773633+0000 mgr.smithi153.asomxr (mgr.14195) 1314 : audit 0 from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:46:09.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:09 smithi153 bash[14340]: cluster 2024-03-12T02:46:07.831829+0000 mgr.smithi153.asomxr (mgr.14195) 1315 : cluster 0 pgmap v1096: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:09.658 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:46:09.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:09 smithi163 bash[22347]: audit 2024-03-12T02:46:07.773633+0000 mgr.smithi153.asomxr (mgr.14195) 1314 : audit 0 from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:46:09.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:09 smithi163 bash[22347]: cluster 2024-03-12T02:46:07.831829+0000 mgr.smithi153.asomxr (mgr.14195) 1315 : cluster 0 pgmap v1096: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:11 smithi163 bash[22347]: cluster 2024-03-12T02:46:09.832448+0000 mgr.smithi153.asomxr (mgr.14195) 1316 : cluster 0 pgmap v1097: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:11.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:11 smithi153 bash[14340]: cluster 2024-03-12T02:46:09.832448+0000 mgr.smithi153.asomxr (mgr.14195) 1316 : cluster 0 pgmap v1097: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:13.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:13 smithi153 bash[14340]: cluster 2024-03-12T02:46:11.833055+0000 mgr.smithi153.asomxr (mgr.14195) 1317 : cluster 0 pgmap v1098: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:13.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:13 smithi163 bash[22347]: cluster 2024-03-12T02:46:11.833055+0000 mgr.smithi153.asomxr (mgr.14195) 1317 : cluster 0 pgmap v1098: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:14.410 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:46:14.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:14 smithi153 bash[14340]: cluster 2024-03-12T02:46:13.833674+0000 mgr.smithi153.asomxr (mgr.14195) 1318 : cluster 0 pgmap v1099: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:14.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:14 smithi163 bash[22347]: cluster 2024-03-12T02:46:13.833674+0000 mgr.smithi153.asomxr (mgr.14195) 1318 : cluster 0 pgmap v1099: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:15.708 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:15 smithi153 bash[14340]: audit 2024-03-12T02:46:15.432481+0000 mon.smithi153 (mon.0) 1111 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:46:15.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:15 smithi163 bash[22347]: audit 2024-03-12T02:46:15.432481+0000 mon.smithi153 (mon.0) 1111 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:46:16.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:16 smithi153 bash[14340]: cluster 2024-03-12T02:46:15.834179+0000 mgr.smithi153.asomxr (mgr.14195) 1319 : cluster 0 pgmap v1100: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:16.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:16 smithi163 bash[22347]: cluster 2024-03-12T02:46:15.834179+0000 mgr.smithi153.asomxr (mgr.14195) 1319 : cluster 0 pgmap v1100: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:17.699 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:46:17.700 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:46:18.531 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:46:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:19 smithi163 bash[22347]: audit 2024-03-12T02:46:17.684875+0000 mgr.smithi153.asomxr (mgr.14195) 1320 : audit 0 from='client.15190 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:46:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:19 smithi163 bash[22347]: cluster 2024-03-12T02:46:17.834566+0000 mgr.smithi153.asomxr (mgr.14195) 1321 : cluster 0 pgmap v1101: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:19.532 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:46:19.543 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:19 smithi153 bash[14340]: audit 2024-03-12T02:46:17.684875+0000 mgr.smithi153.asomxr (mgr.14195) 1320 : audit 0 from='client.15190 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:46:19.543 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:19 smithi153 bash[14340]: cluster 2024-03-12T02:46:17.834566+0000 mgr.smithi153.asomxr (mgr.14195) 1321 : cluster 0 pgmap v1101: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:21 smithi163 bash[22347]: cluster 2024-03-12T02:46:19.835196+0000 mgr.smithi153.asomxr (mgr.14195) 1322 : cluster 0 pgmap v1102: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:21.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:21 smithi153 bash[14340]: cluster 2024-03-12T02:46:19.835196+0000 mgr.smithi153.asomxr (mgr.14195) 1322 : cluster 0 pgmap v1102: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:23.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:23 smithi153 bash[14340]: cluster 2024-03-12T02:46:21.835841+0000 mgr.smithi153.asomxr (mgr.14195) 1323 : cluster 0 pgmap v1103: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:23.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:23 smithi163 bash[22347]: cluster 2024-03-12T02:46:21.835841+0000 mgr.smithi153.asomxr (mgr.14195) 1323 : cluster 0 pgmap v1103: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:24.281 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:46:25.464 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:25 smithi153 bash[14340]: cluster 2024-03-12T02:46:23.836510+0000 mgr.smithi153.asomxr (mgr.14195) 1324 : cluster 0 pgmap v1104: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:25.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:25 smithi163 bash[22347]: cluster 2024-03-12T02:46:23.836510+0000 mgr.smithi153.asomxr (mgr.14195) 1324 : cluster 0 pgmap v1104: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:27.386 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:27 smithi153 bash[14340]: cluster 2024-03-12T02:46:25.837244+0000 mgr.smithi153.asomxr (mgr.14195) 1325 : cluster 0 pgmap v1105: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:27.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:27 smithi163 bash[22347]: cluster 2024-03-12T02:46:25.837244+0000 mgr.smithi153.asomxr (mgr.14195) 1325 : cluster 0 pgmap v1105: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:27.543 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:46:27.544 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:46:28.358 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:46:29.359 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:46:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:29 smithi163 bash[22347]: audit 2024-03-12T02:46:27.529149+0000 mgr.smithi153.asomxr (mgr.14195) 1326 : audit 0 from='client.15194 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:46:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:29 smithi163 bash[22347]: cluster 2024-03-12T02:46:27.837843+0000 mgr.smithi153.asomxr (mgr.14195) 1327 : cluster 0 pgmap v1106: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:29.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:29 smithi153 bash[14340]: audit 2024-03-12T02:46:27.529149+0000 mgr.smithi153.asomxr (mgr.14195) 1326 : audit 0 from='client.15194 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:46:29.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:29 smithi153 bash[14340]: cluster 2024-03-12T02:46:27.837843+0000 mgr.smithi153.asomxr (mgr.14195) 1327 : cluster 0 pgmap v1106: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:31 smithi163 bash[22347]: cluster 2024-03-12T02:46:29.838485+0000 mgr.smithi153.asomxr (mgr.14195) 1328 : cluster 0 pgmap v1107: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:31 smithi163 bash[22347]: audit 2024-03-12T02:46:30.432930+0000 mon.smithi153 (mon.0) 1112 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:46:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:31 smithi153 bash[14340]: cluster 2024-03-12T02:46:29.838485+0000 mgr.smithi153.asomxr (mgr.14195) 1328 : cluster 0 pgmap v1107: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:31 smithi153 bash[14340]: audit 2024-03-12T02:46:30.432930+0000 mon.smithi153 (mon.0) 1112 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:46:33.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:33 smithi163 bash[22347]: cluster 2024-03-12T02:46:31.839145+0000 mgr.smithi153.asomxr (mgr.14195) 1329 : cluster 0 pgmap v1108: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:33.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:33 smithi153 bash[14340]: cluster 2024-03-12T02:46:31.839145+0000 mgr.smithi153.asomxr (mgr.14195) 1329 : cluster 0 pgmap v1108: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:34.115 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:46:35.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:35 smithi163 bash[22347]: cluster 2024-03-12T02:46:33.839734+0000 mgr.smithi153.asomxr (mgr.14195) 1330 : cluster 0 pgmap v1109: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:35.487 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:35 smithi153 bash[14340]: cluster 2024-03-12T02:46:33.839734+0000 mgr.smithi153.asomxr (mgr.14195) 1330 : cluster 0 pgmap v1109: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:37.405 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:46:37.406 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:46:37.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:37 smithi163 bash[22347]: cluster 2024-03-12T02:46:35.840244+0000 mgr.smithi153.asomxr (mgr.14195) 1331 : cluster 0 pgmap v1110: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:37.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:37 smithi153 bash[14340]: cluster 2024-03-12T02:46:35.840244+0000 mgr.smithi153.asomxr (mgr.14195) 1331 : cluster 0 pgmap v1110: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:38.197 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:46:39.198 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:46:39.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:39 smithi163 bash[22347]: audit 2024-03-12T02:46:37.390957+0000 mgr.smithi153.asomxr (mgr.14195) 1332 : audit 0 from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:46:39.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:39 smithi163 bash[22347]: cluster 2024-03-12T02:46:37.840816+0000 mgr.smithi153.asomxr (mgr.14195) 1333 : cluster 0 pgmap v1111: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:39.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:39 smithi153 bash[14340]: audit 2024-03-12T02:46:37.390957+0000 mgr.smithi153.asomxr (mgr.14195) 1332 : audit 0 from='client.15198 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:46:39.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:39 smithi153 bash[14340]: cluster 2024-03-12T02:46:37.840816+0000 mgr.smithi153.asomxr (mgr.14195) 1333 : cluster 0 pgmap v1111: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:40.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:40 smithi153 bash[14340]: audit 2024-03-12T02:46:39.931644+0000 mon.smithi153 (mon.0) 1113 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:46:40.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:40 smithi163 bash[22347]: audit 2024-03-12T02:46:39.931644+0000 mon.smithi153 (mon.0) 1113 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:46:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:41 smithi153 bash[14340]: cluster 2024-03-12T02:46:39.841464+0000 mgr.smithi153.asomxr (mgr.14195) 1334 : cluster 0 pgmap v1112: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:41 smithi153 bash[14340]: audit 2024-03-12T02:46:40.509835+0000 mon.smithi153 (mon.0) 1114 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:46:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:41 smithi153 bash[14340]: audit 2024-03-12T02:46:40.511393+0000 mon.smithi153 (mon.0) 1115 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:46:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:41 smithi153 bash[14340]: audit 2024-03-12T02:46:40.512756+0000 mon.smithi153 (mon.0) 1116 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:46:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:41 smithi153 bash[14340]: audit 2024-03-12T02:46:40.514171+0000 mon.smithi153 (mon.0) 1117 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:46:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:41 smithi153 bash[14340]: audit 2024-03-12T02:46:40.522185+0000 mon.smithi153 (mon.0) 1118 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:46:41.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:41 smithi153 bash[14340]: audit 2024-03-12T02:46:40.523396+0000 mon.smithi153 (mon.0) 1119 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:46:41.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:41 smithi153 bash[14340]: audit 2024-03-12T02:46:40.524515+0000 mon.smithi153 (mon.0) 1120 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:46:41.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:41 smithi153 bash[14340]: audit 2024-03-12T02:46:40.531622+0000 mon.smithi153 (mon.0) 1121 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:46:41.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:41 smithi153 bash[14340]: audit 2024-03-12T02:46:40.536405+0000 mon.smithi153 (mon.0) 1122 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:46:41.720 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:41 smithi163 bash[22347]: cluster 2024-03-12T02:46:39.841464+0000 mgr.smithi153.asomxr (mgr.14195) 1334 : cluster 0 pgmap v1112: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:41.720 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:41 smithi163 bash[22347]: audit 2024-03-12T02:46:40.509835+0000 mon.smithi153 (mon.0) 1114 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:46:41.720 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:41 smithi163 bash[22347]: audit 2024-03-12T02:46:40.511393+0000 mon.smithi153 (mon.0) 1115 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:46:41.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:41 smithi163 bash[22347]: audit 2024-03-12T02:46:40.512756+0000 mon.smithi153 (mon.0) 1116 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:46:41.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:41 smithi163 bash[22347]: audit 2024-03-12T02:46:40.514171+0000 mon.smithi153 (mon.0) 1117 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-12T02:46:41.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:41 smithi163 bash[22347]: audit 2024-03-12T02:46:40.522185+0000 mon.smithi153 (mon.0) 1118 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:46:41.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:41 smithi163 bash[22347]: audit 2024-03-12T02:46:40.523396+0000 mon.smithi153 (mon.0) 1119 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:46:41.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:41 smithi163 bash[22347]: audit 2024-03-12T02:46:40.524515+0000 mon.smithi153 (mon.0) 1120 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:46:41.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:41 smithi163 bash[22347]: audit 2024-03-12T02:46:40.531622+0000 mon.smithi153 (mon.0) 1121 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:46:41.721 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:41 smithi163 bash[22347]: audit 2024-03-12T02:46:40.536405+0000 mon.smithi153 (mon.0) 1122 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:46:42.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:42 smithi153 bash[14340]: cephadm 2024-03-12T02:46:40.515245+0000 mgr.smithi153.asomxr (mgr.14195) 1335 : cephadm 1 Adjusting osd_memory_target on smithi163 to 2929M 2024-03-12T02:46:42.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:42 smithi153 bash[14340]: cluster 2024-03-12T02:46:40.526677+0000 mgr.smithi153.asomxr (mgr.14195) 1336 : cluster 0 pgmap v1113: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:42.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:42 smithi153 bash[14340]: cluster 2024-03-12T02:46:40.527069+0000 mgr.smithi153.asomxr (mgr.14195) 1337 : cluster 0 pgmap v1114: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:42 smithi163 bash[22347]: cephadm 2024-03-12T02:46:40.515245+0000 mgr.smithi153.asomxr (mgr.14195) 1335 : cephadm 1 Adjusting osd_memory_target on smithi163 to 2929M 2024-03-12T02:46:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:42 smithi163 bash[22347]: cluster 2024-03-12T02:46:40.526677+0000 mgr.smithi153.asomxr (mgr.14195) 1336 : cluster 0 pgmap v1113: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:42 smithi163 bash[22347]: cluster 2024-03-12T02:46:40.527069+0000 mgr.smithi153.asomxr (mgr.14195) 1337 : cluster 0 pgmap v1114: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:43.948 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:46:44.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:44 smithi153 bash[14340]: cluster 2024-03-12T02:46:42.527658+0000 mgr.smithi153.asomxr (mgr.14195) 1338 : cluster 0 pgmap v1115: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:44.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:44 smithi163 bash[22347]: cluster 2024-03-12T02:46:42.527658+0000 mgr.smithi153.asomxr (mgr.14195) 1338 : cluster 0 pgmap v1115: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:46.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:46 smithi153 bash[14340]: cluster 2024-03-12T02:46:44.528286+0000 mgr.smithi153.asomxr (mgr.14195) 1339 : cluster 0 pgmap v1116: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:46 smithi153 bash[14340]: audit 2024-03-12T02:46:45.432701+0000 mon.smithi153 (mon.0) 1123 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:46:46.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:46 smithi163 bash[22347]: cluster 2024-03-12T02:46:44.528286+0000 mgr.smithi153.asomxr (mgr.14195) 1339 : cluster 0 pgmap v1116: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:46.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:46 smithi163 bash[22347]: audit 2024-03-12T02:46:45.432701+0000 mon.smithi153 (mon.0) 1123 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:46:47.341 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:46:47.342 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:46:48.206 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:46:48.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:48 smithi153 bash[14340]: cluster 2024-03-12T02:46:46.528901+0000 mgr.smithi153.asomxr (mgr.14195) 1340 : cluster 0 pgmap v1117: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:48.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:48 smithi163 bash[22347]: cluster 2024-03-12T02:46:46.528901+0000 mgr.smithi153.asomxr (mgr.14195) 1340 : cluster 0 pgmap v1117: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:49.207 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:46:49.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:49 smithi153 bash[14340]: audit 2024-03-12T02:46:47.326606+0000 mgr.smithi153.asomxr (mgr.14195) 1341 : audit 0 from='client.15202 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:46:49.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:49 smithi163 bash[22347]: audit 2024-03-12T02:46:47.326606+0000 mgr.smithi153.asomxr (mgr.14195) 1341 : audit 0 from='client.15202 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:46:50.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:50 smithi153 bash[14340]: cluster 2024-03-12T02:46:48.529613+0000 mgr.smithi153.asomxr (mgr.14195) 1342 : cluster 0 pgmap v1118: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:50.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:50 smithi163 bash[22347]: cluster 2024-03-12T02:46:48.529613+0000 mgr.smithi153.asomxr (mgr.14195) 1342 : cluster 0 pgmap v1118: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:51.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:51 smithi153 bash[14340]: cluster 2024-03-12T02:46:50.530206+0000 mgr.smithi153.asomxr (mgr.14195) 1343 : cluster 0 pgmap v1119: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:51.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:51 smithi163 bash[22347]: cluster 2024-03-12T02:46:50.530206+0000 mgr.smithi153.asomxr (mgr.14195) 1343 : cluster 0 pgmap v1119: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:53.967 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:46:54.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:54 smithi153 bash[14340]: cluster 2024-03-12T02:46:52.530776+0000 mgr.smithi153.asomxr (mgr.14195) 1344 : cluster 0 pgmap v1120: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:54 smithi163 bash[22347]: cluster 2024-03-12T02:46:52.530776+0000 mgr.smithi153.asomxr (mgr.14195) 1344 : cluster 0 pgmap v1120: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:56.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:56 smithi153 bash[14340]: cluster 2024-03-12T02:46:54.531431+0000 mgr.smithi153.asomxr (mgr.14195) 1345 : cluster 0 pgmap v1121: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:56.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:56 smithi163 bash[22347]: cluster 2024-03-12T02:46:54.531431+0000 mgr.smithi153.asomxr (mgr.14195) 1345 : cluster 0 pgmap v1121: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:57.312 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:46:57.312 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:46:58.148 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:46:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:58 smithi153 bash[14340]: cluster 2024-03-12T02:46:56.532157+0000 mgr.smithi153.asomxr (mgr.14195) 1346 : cluster 0 pgmap v1122: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:58.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:58 smithi163 bash[22347]: cluster 2024-03-12T02:46:56.532157+0000 mgr.smithi153.asomxr (mgr.14195) 1346 : cluster 0 pgmap v1122: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:46:59.149 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:46:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:46:59 smithi163 bash[22347]: audit 2024-03-12T02:46:57.301050+0000 mgr.smithi153.asomxr (mgr.14195) 1347 : audit 0 from='client.15206 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:46:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:46:59 smithi153 bash[14340]: audit 2024-03-12T02:46:57.301050+0000 mgr.smithi153.asomxr (mgr.14195) 1347 : audit 0 from='client.15206 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:00.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:00 smithi163 bash[22347]: cluster 2024-03-12T02:46:58.532888+0000 mgr.smithi153.asomxr (mgr.14195) 1348 : cluster 0 pgmap v1123: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:00.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:00 smithi153 bash[14340]: cluster 2024-03-12T02:46:58.532888+0000 mgr.smithi153.asomxr (mgr.14195) 1348 : cluster 0 pgmap v1123: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:01 smithi163 bash[22347]: audit 2024-03-12T02:47:00.433433+0000 mon.smithi153 (mon.0) 1124 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:47:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:01 smithi153 bash[14340]: audit 2024-03-12T02:47:00.433433+0000 mon.smithi153 (mon.0) 1124 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:47:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:02 smithi163 bash[22347]: cluster 2024-03-12T02:47:00.533571+0000 mgr.smithi153.asomxr (mgr.14195) 1349 : cluster 0 pgmap v1124: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:02.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:02 smithi153 bash[14340]: cluster 2024-03-12T02:47:00.533571+0000 mgr.smithi153.asomxr (mgr.14195) 1349 : cluster 0 pgmap v1124: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:03.898 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:47:04.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:04 smithi153 bash[14340]: cluster 2024-03-12T02:47:02.534144+0000 mgr.smithi153.asomxr (mgr.14195) 1350 : cluster 0 pgmap v1125: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:04.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:04 smithi163 bash[22347]: cluster 2024-03-12T02:47:02.534144+0000 mgr.smithi153.asomxr (mgr.14195) 1350 : cluster 0 pgmap v1125: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:06.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:06 smithi153 bash[14340]: cluster 2024-03-12T02:47:04.534729+0000 mgr.smithi153.asomxr (mgr.14195) 1351 : cluster 0 pgmap v1126: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:06.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:06 smithi163 bash[22347]: cluster 2024-03-12T02:47:04.534729+0000 mgr.smithi153.asomxr (mgr.14195) 1351 : cluster 0 pgmap v1126: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:07.173 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:47:07.174 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:47:07.929 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:47:08.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:08 smithi153 bash[14340]: cluster 2024-03-12T02:47:06.535352+0000 mgr.smithi153.asomxr (mgr.14195) 1352 : cluster 0 pgmap v1127: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:08.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:08 smithi153 bash[14340]: audit 2024-03-12T02:47:07.163180+0000 mgr.smithi153.asomxr (mgr.14195) 1353 : audit 0 from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:08.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:08 smithi163 bash[22347]: cluster 2024-03-12T02:47:06.535352+0000 mgr.smithi153.asomxr (mgr.14195) 1352 : cluster 0 pgmap v1127: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:08.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:08 smithi163 bash[22347]: audit 2024-03-12T02:47:07.163180+0000 mgr.smithi153.asomxr (mgr.14195) 1353 : audit 0 from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:08.930 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:47:10.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:10 smithi153 bash[14340]: cluster 2024-03-12T02:47:08.536004+0000 mgr.smithi153.asomxr (mgr.14195) 1354 : cluster 0 pgmap v1128: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:10.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:10 smithi163 bash[22347]: cluster 2024-03-12T02:47:08.536004+0000 mgr.smithi153.asomxr (mgr.14195) 1354 : cluster 0 pgmap v1128: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:12.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:12 smithi153 bash[14340]: cluster 2024-03-12T02:47:10.536657+0000 mgr.smithi153.asomxr (mgr.14195) 1355 : cluster 0 pgmap v1129: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:12.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:12 smithi163 bash[22347]: cluster 2024-03-12T02:47:10.536657+0000 mgr.smithi153.asomxr (mgr.14195) 1355 : cluster 0 pgmap v1129: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:13.680 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:47:14.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:14 smithi153 bash[14340]: cluster 2024-03-12T02:47:12.537335+0000 mgr.smithi153.asomxr (mgr.14195) 1356 : cluster 0 pgmap v1130: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:14.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:14 smithi163 bash[22347]: cluster 2024-03-12T02:47:12.537335+0000 mgr.smithi153.asomxr (mgr.14195) 1356 : cluster 0 pgmap v1130: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:16.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:16 smithi153 bash[14340]: cluster 2024-03-12T02:47:14.537939+0000 mgr.smithi153.asomxr (mgr.14195) 1357 : cluster 0 pgmap v1131: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:16 smithi153 bash[14340]: audit 2024-03-12T02:47:15.433930+0000 mon.smithi153 (mon.0) 1125 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:47:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:16 smithi163 bash[22347]: cluster 2024-03-12T02:47:14.537939+0000 mgr.smithi153.asomxr (mgr.14195) 1357 : cluster 0 pgmap v1131: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:16 smithi163 bash[22347]: audit 2024-03-12T02:47:15.433930+0000 mon.smithi153 (mon.0) 1125 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:47:17.043 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:47:17.043 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:47:17.845 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:47:18.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:18 smithi153 bash[14340]: cluster 2024-03-12T02:47:16.538583+0000 mgr.smithi153.asomxr (mgr.14195) 1358 : cluster 0 pgmap v1132: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:18.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:18 smithi153 bash[14340]: audit 2024-03-12T02:47:17.031598+0000 mgr.smithi153.asomxr (mgr.14195) 1359 : audit 0 from='client.15214 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:18.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:18 smithi163 bash[22347]: cluster 2024-03-12T02:47:16.538583+0000 mgr.smithi153.asomxr (mgr.14195) 1358 : cluster 0 pgmap v1132: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:18.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:18 smithi163 bash[22347]: audit 2024-03-12T02:47:17.031598+0000 mgr.smithi153.asomxr (mgr.14195) 1359 : audit 0 from='client.15214 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:18.846 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:47:20.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:20 smithi153 bash[14340]: cluster 2024-03-12T02:47:18.539210+0000 mgr.smithi153.asomxr (mgr.14195) 1360 : cluster 0 pgmap v1133: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:20.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:20 smithi163 bash[22347]: cluster 2024-03-12T02:47:18.539210+0000 mgr.smithi153.asomxr (mgr.14195) 1360 : cluster 0 pgmap v1133: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:21.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:21 smithi153 bash[14340]: cluster 2024-03-12T02:47:20.539866+0000 mgr.smithi153.asomxr (mgr.14195) 1361 : cluster 0 pgmap v1134: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:21.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:21 smithi163 bash[22347]: cluster 2024-03-12T02:47:20.539866+0000 mgr.smithi153.asomxr (mgr.14195) 1361 : cluster 0 pgmap v1134: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:23.595 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:47:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:24 smithi163 bash[22347]: cluster 2024-03-12T02:47:22.540563+0000 mgr.smithi153.asomxr (mgr.14195) 1362 : cluster 0 pgmap v1135: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:24.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:24 smithi153 bash[14340]: cluster 2024-03-12T02:47:22.540563+0000 mgr.smithi153.asomxr (mgr.14195) 1362 : cluster 0 pgmap v1135: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:26 smithi163 bash[22347]: cluster 2024-03-12T02:47:24.541251+0000 mgr.smithi153.asomxr (mgr.14195) 1363 : cluster 0 pgmap v1136: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:26.488 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:26 smithi153 bash[14340]: cluster 2024-03-12T02:47:24.541251+0000 mgr.smithi153.asomxr (mgr.14195) 1363 : cluster 0 pgmap v1136: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:26.899 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:47:26.899 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:47:27.678 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:47:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:28 smithi163 bash[22347]: cluster 2024-03-12T02:47:26.541842+0000 mgr.smithi153.asomxr (mgr.14195) 1364 : cluster 0 pgmap v1137: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:28 smithi163 bash[22347]: audit 2024-03-12T02:47:26.883806+0000 mgr.smithi153.asomxr (mgr.14195) 1365 : audit 0 from='client.15218 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:28.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:28 smithi153 bash[14340]: cluster 2024-03-12T02:47:26.541842+0000 mgr.smithi153.asomxr (mgr.14195) 1364 : cluster 0 pgmap v1137: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:28.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:28 smithi153 bash[14340]: audit 2024-03-12T02:47:26.883806+0000 mgr.smithi153.asomxr (mgr.14195) 1365 : audit 0 from='client.15218 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:28.679 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:47:30.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:30 smithi163 bash[22347]: cluster 2024-03-12T02:47:28.542484+0000 mgr.smithi153.asomxr (mgr.14195) 1366 : cluster 0 pgmap v1138: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:30.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:30 smithi153 bash[14340]: cluster 2024-03-12T02:47:28.542484+0000 mgr.smithi153.asomxr (mgr.14195) 1366 : cluster 0 pgmap v1138: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:31.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:31 smithi163 bash[22347]: audit 2024-03-12T02:47:30.434434+0000 mon.smithi153 (mon.0) 1126 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:47:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:31 smithi153 bash[14340]: audit 2024-03-12T02:47:30.434434+0000 mon.smithi153 (mon.0) 1126 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:47:32.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:32 smithi163 bash[22347]: cluster 2024-03-12T02:47:30.543172+0000 mgr.smithi153.asomxr (mgr.14195) 1367 : cluster 0 pgmap v1139: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:32.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:32 smithi153 bash[14340]: cluster 2024-03-12T02:47:30.543172+0000 mgr.smithi153.asomxr (mgr.14195) 1367 : cluster 0 pgmap v1139: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:33.438 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:47:34.485 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:34 smithi153 bash[14340]: cluster 2024-03-12T02:47:32.543808+0000 mgr.smithi153.asomxr (mgr.14195) 1368 : cluster 0 pgmap v1140: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:34.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:34 smithi163 bash[22347]: cluster 2024-03-12T02:47:32.543808+0000 mgr.smithi153.asomxr (mgr.14195) 1368 : cluster 0 pgmap v1140: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:36.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:36 smithi153 bash[14340]: cluster 2024-03-12T02:47:34.544274+0000 mgr.smithi153.asomxr (mgr.14195) 1369 : cluster 0 pgmap v1141: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:36.638 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:47:36.638 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:47:36.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:36 smithi163 bash[22347]: cluster 2024-03-12T02:47:34.544274+0000 mgr.smithi153.asomxr (mgr.14195) 1369 : cluster 0 pgmap v1141: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:37.457 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:47:38.458 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:47:38.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:38 smithi153 bash[14340]: cluster 2024-03-12T02:47:36.544799+0000 mgr.smithi153.asomxr (mgr.14195) 1370 : cluster 0 pgmap v1142: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:38.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:38 smithi153 bash[14340]: audit 2024-03-12T02:47:36.622753+0000 mgr.smithi153.asomxr (mgr.14195) 1371 : audit 0 from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:38.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:38 smithi163 bash[22347]: cluster 2024-03-12T02:47:36.544799+0000 mgr.smithi153.asomxr (mgr.14195) 1370 : cluster 0 pgmap v1142: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:38.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:38 smithi163 bash[22347]: audit 2024-03-12T02:47:36.622753+0000 mgr.smithi153.asomxr (mgr.14195) 1371 : audit 0 from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:40.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:40 smithi153 bash[14340]: cluster 2024-03-12T02:47:38.545444+0000 mgr.smithi153.asomxr (mgr.14195) 1372 : cluster 0 pgmap v1143: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:40.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:40 smithi163 bash[22347]: cluster 2024-03-12T02:47:38.545444+0000 mgr.smithi153.asomxr (mgr.14195) 1372 : cluster 0 pgmap v1143: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:41 smithi153 bash[14340]: audit 2024-03-12T02:47:40.627887+0000 mon.smithi153 (mon.0) 1127 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:47:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:41 smithi153 bash[14340]: audit 2024-03-12T02:47:41.216944+0000 mon.smithi153 (mon.0) 1128 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:47:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:41 smithi153 bash[14340]: audit 2024-03-12T02:47:41.218498+0000 mon.smithi153 (mon.0) 1129 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:47:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:41 smithi153 bash[14340]: audit 2024-03-12T02:47:41.227645+0000 mon.smithi153 (mon.0) 1130 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:47:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:41 smithi153 bash[14340]: audit 2024-03-12T02:47:41.232569+0000 mon.smithi153 (mon.0) 1131 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:47:41.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:41 smithi163 bash[22347]: audit 2024-03-12T02:47:40.627887+0000 mon.smithi153 (mon.0) 1127 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:47:41.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:41 smithi163 bash[22347]: audit 2024-03-12T02:47:41.216944+0000 mon.smithi153 (mon.0) 1128 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:47:41.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:41 smithi163 bash[22347]: audit 2024-03-12T02:47:41.218498+0000 mon.smithi153 (mon.0) 1129 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:47:41.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:41 smithi163 bash[22347]: audit 2024-03-12T02:47:41.227645+0000 mon.smithi153 (mon.0) 1130 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:47:41.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:41 smithi163 bash[22347]: audit 2024-03-12T02:47:41.232569+0000 mon.smithi153 (mon.0) 1131 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:47:42.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:42 smithi153 bash[14340]: cluster 2024-03-12T02:47:40.546044+0000 mgr.smithi153.asomxr (mgr.14195) 1373 : cluster 0 pgmap v1144: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:42.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:42 smithi153 bash[14340]: cluster 2024-03-12T02:47:41.220942+0000 mgr.smithi153.asomxr (mgr.14195) 1374 : cluster 0 pgmap v1145: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:42.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:42 smithi153 bash[14340]: cluster 2024-03-12T02:47:41.221392+0000 mgr.smithi153.asomxr (mgr.14195) 1375 : cluster 0 pgmap v1146: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:42 smithi163 bash[22347]: cluster 2024-03-12T02:47:40.546044+0000 mgr.smithi153.asomxr (mgr.14195) 1373 : cluster 0 pgmap v1144: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:42 smithi163 bash[22347]: cluster 2024-03-12T02:47:41.220942+0000 mgr.smithi153.asomxr (mgr.14195) 1374 : cluster 0 pgmap v1145: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:42 smithi163 bash[22347]: cluster 2024-03-12T02:47:41.221392+0000 mgr.smithi153.asomxr (mgr.14195) 1375 : cluster 0 pgmap v1146: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:43.209 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:47:44.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:44 smithi163 bash[22347]: cluster 2024-03-12T02:47:43.222052+0000 mgr.smithi153.asomxr (mgr.14195) 1376 : cluster 0 pgmap v1147: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:44.510 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:44 smithi153 bash[14340]: cluster 2024-03-12T02:47:43.222052+0000 mgr.smithi153.asomxr (mgr.14195) 1376 : cluster 0 pgmap v1147: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:45.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:45 smithi153 bash[14340]: cluster 2024-03-12T02:47:45.222849+0000 mgr.smithi153.asomxr (mgr.14195) 1377 : cluster 0 pgmap v1148: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:45.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:45 smithi163 bash[22347]: cluster 2024-03-12T02:47:45.222849+0000 mgr.smithi153.asomxr (mgr.14195) 1377 : cluster 0 pgmap v1148: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:46.536 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:47:46.536 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:47:46.592 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:46 smithi153 bash[14340]: audit 2024-03-12T02:47:45.434360+0000 mon.smithi153 (mon.0) 1132 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:47:46.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:46 smithi163 bash[22347]: audit 2024-03-12T02:47:45.434360+0000 mon.smithi153 (mon.0) 1132 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:47:47.385 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:47:47.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:47 smithi163 bash[22347]: audit 2024-03-12T02:47:46.525646+0000 mgr.smithi153.asomxr (mgr.14195) 1378 : audit 0 from='client.15226 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:47.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:47 smithi163 bash[22347]: cluster 2024-03-12T02:47:47.223497+0000 mgr.smithi153.asomxr (mgr.14195) 1379 : cluster 0 pgmap v1149: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:47.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:47 smithi153 bash[14340]: audit 2024-03-12T02:47:46.525646+0000 mgr.smithi153.asomxr (mgr.14195) 1378 : audit 0 from='client.15226 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:47.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:47 smithi153 bash[14340]: cluster 2024-03-12T02:47:47.223497+0000 mgr.smithi153.asomxr (mgr.14195) 1379 : cluster 0 pgmap v1149: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:48.386 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:47:49.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:49 smithi163 bash[22347]: cluster 2024-03-12T02:47:49.224090+0000 mgr.smithi153.asomxr (mgr.14195) 1380 : cluster 0 pgmap v1150: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:49.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:49 smithi153 bash[14340]: cluster 2024-03-12T02:47:49.224090+0000 mgr.smithi153.asomxr (mgr.14195) 1380 : cluster 0 pgmap v1150: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:51.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:51 smithi163 bash[22347]: cluster 2024-03-12T02:47:51.224725+0000 mgr.smithi153.asomxr (mgr.14195) 1381 : cluster 0 pgmap v1151: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:51.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:51 smithi153 bash[14340]: cluster 2024-03-12T02:47:51.224725+0000 mgr.smithi153.asomxr (mgr.14195) 1381 : cluster 0 pgmap v1151: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:53.137 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:47:54.320 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:54 smithi153 bash[14340]: cluster 2024-03-12T02:47:53.225224+0000 mgr.smithi153.asomxr (mgr.14195) 1382 : cluster 0 pgmap v1152: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:54 smithi163 bash[22347]: cluster 2024-03-12T02:47:53.225224+0000 mgr.smithi153.asomxr (mgr.14195) 1382 : cluster 0 pgmap v1152: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:55.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:55 smithi153 bash[14340]: cluster 2024-03-12T02:47:55.225759+0000 mgr.smithi153.asomxr (mgr.14195) 1383 : cluster 0 pgmap v1153: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:55.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:55 smithi163 bash[22347]: cluster 2024-03-12T02:47:55.225759+0000 mgr.smithi153.asomxr (mgr.14195) 1383 : cluster 0 pgmap v1153: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:56.416 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:47:56.416 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:47:57.244 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:47:57.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:57 smithi153 bash[14340]: audit 2024-03-12T02:47:56.405717+0000 mgr.smithi153.asomxr (mgr.14195) 1384 : audit 0 from='client.15230 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:57.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:57 smithi153 bash[14340]: cluster 2024-03-12T02:47:57.226306+0000 mgr.smithi153.asomxr (mgr.14195) 1385 : cluster 0 pgmap v1154: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:57.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:57 smithi163 bash[22347]: audit 2024-03-12T02:47:56.405717+0000 mgr.smithi153.asomxr (mgr.14195) 1384 : audit 0 from='client.15230 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:47:57.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:57 smithi163 bash[22347]: cluster 2024-03-12T02:47:57.226306+0000 mgr.smithi153.asomxr (mgr.14195) 1385 : cluster 0 pgmap v1154: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:58.246 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:47:59.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:47:59 smithi153 bash[14340]: cluster 2024-03-12T02:47:59.226921+0000 mgr.smithi153.asomxr (mgr.14195) 1386 : cluster 0 pgmap v1155: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:47:59.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:47:59 smithi163 bash[22347]: cluster 2024-03-12T02:47:59.226921+0000 mgr.smithi153.asomxr (mgr.14195) 1386 : cluster 0 pgmap v1155: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:00.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:00 smithi153 bash[14340]: audit 2024-03-12T02:48:00.435025+0000 mon.smithi153 (mon.0) 1133 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:48:00.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:00 smithi163 bash[22347]: audit 2024-03-12T02:48:00.435025+0000 mon.smithi153 (mon.0) 1133 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:48:01.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:01 smithi153 bash[14340]: cluster 2024-03-12T02:48:01.227575+0000 mgr.smithi153.asomxr (mgr.14195) 1387 : cluster 0 pgmap v1156: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:01.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:01 smithi163 bash[22347]: cluster 2024-03-12T02:48:01.227575+0000 mgr.smithi153.asomxr (mgr.14195) 1387 : cluster 0 pgmap v1156: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:02.999 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:48:04.314 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:04 smithi153 bash[14340]: cluster 2024-03-12T02:48:03.228117+0000 mgr.smithi153.asomxr (mgr.14195) 1388 : cluster 0 pgmap v1157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:04 smithi163 bash[22347]: cluster 2024-03-12T02:48:03.228117+0000 mgr.smithi153.asomxr (mgr.14195) 1388 : cluster 0 pgmap v1157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:05.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:05 smithi163 bash[22347]: cluster 2024-03-12T02:48:05.228747+0000 mgr.smithi153.asomxr (mgr.14195) 1389 : cluster 0 pgmap v1158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:05.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:05 smithi153 bash[14340]: cluster 2024-03-12T02:48:05.228747+0000 mgr.smithi153.asomxr (mgr.14195) 1389 : cluster 0 pgmap v1158: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:06.285 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:48:06.285 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:48:06.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:06 smithi153 bash[14340]: audit 2024-03-12T02:48:06.270684+0000 mgr.smithi153.asomxr (mgr.14195) 1390 : audit 0 from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:06.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:06 smithi163 bash[22347]: audit 2024-03-12T02:48:06.270684+0000 mgr.smithi153.asomxr (mgr.14195) 1390 : audit 0 from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:07.085 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:48:07.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:07 smithi163 bash[22347]: cluster 2024-03-12T02:48:07.229427+0000 mgr.smithi153.asomxr (mgr.14195) 1391 : cluster 0 pgmap v1159: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:07.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:07 smithi153 bash[14340]: cluster 2024-03-12T02:48:07.229427+0000 mgr.smithi153.asomxr (mgr.14195) 1391 : cluster 0 pgmap v1159: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:08.085 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:48:09.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:09 smithi153 bash[14340]: cluster 2024-03-12T02:48:09.230039+0000 mgr.smithi153.asomxr (mgr.14195) 1392 : cluster 0 pgmap v1160: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:09.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:09 smithi163 bash[22347]: cluster 2024-03-12T02:48:09.230039+0000 mgr.smithi153.asomxr (mgr.14195) 1392 : cluster 0 pgmap v1160: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:11.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:11 smithi163 bash[22347]: cluster 2024-03-12T02:48:11.230688+0000 mgr.smithi153.asomxr (mgr.14195) 1393 : cluster 0 pgmap v1161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:11.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:11 smithi153 bash[14340]: cluster 2024-03-12T02:48:11.230688+0000 mgr.smithi153.asomxr (mgr.14195) 1393 : cluster 0 pgmap v1161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:12.848 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:48:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:14 smithi163 bash[22347]: cluster 2024-03-12T02:48:13.231299+0000 mgr.smithi153.asomxr (mgr.14195) 1394 : cluster 0 pgmap v1162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:14.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:14 smithi153 bash[14340]: cluster 2024-03-12T02:48:13.231299+0000 mgr.smithi153.asomxr (mgr.14195) 1394 : cluster 0 pgmap v1162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:15.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:15 smithi163 bash[22347]: cluster 2024-03-12T02:48:15.231759+0000 mgr.smithi153.asomxr (mgr.14195) 1395 : cluster 0 pgmap v1163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:15.764 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:15 smithi153 bash[14340]: cluster 2024-03-12T02:48:15.231759+0000 mgr.smithi153.asomxr (mgr.14195) 1395 : cluster 0 pgmap v1163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:16.234 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:48:16.234 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:48:16.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:16 smithi153 bash[14340]: audit 2024-03-12T02:48:15.435600+0000 mon.smithi153 (mon.0) 1134 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:48:16.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:16 smithi153 bash[14340]: audit 2024-03-12T02:48:16.218709+0000 mgr.smithi153.asomxr (mgr.14195) 1396 : audit 0 from='client.15238 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:16 smithi163 bash[22347]: audit 2024-03-12T02:48:15.435600+0000 mon.smithi153 (mon.0) 1134 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:48:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:16 smithi163 bash[22347]: audit 2024-03-12T02:48:16.218709+0000 mgr.smithi153.asomxr (mgr.14195) 1396 : audit 0 from='client.15238 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:17.019 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:48:17.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:17 smithi163 bash[22347]: cluster 2024-03-12T02:48:17.232382+0000 mgr.smithi153.asomxr (mgr.14195) 1397 : cluster 0 pgmap v1164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:17.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:17 smithi153 bash[14340]: cluster 2024-03-12T02:48:17.232382+0000 mgr.smithi153.asomxr (mgr.14195) 1397 : cluster 0 pgmap v1164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:18.020 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:48:19.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:19 smithi163 bash[22347]: cluster 2024-03-12T02:48:19.233011+0000 mgr.smithi153.asomxr (mgr.14195) 1398 : cluster 0 pgmap v1165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:19.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:19 smithi153 bash[14340]: cluster 2024-03-12T02:48:19.233011+0000 mgr.smithi153.asomxr (mgr.14195) 1398 : cluster 0 pgmap v1165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:21.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:21 smithi163 bash[22347]: cluster 2024-03-12T02:48:21.233771+0000 mgr.smithi153.asomxr (mgr.14195) 1399 : cluster 0 pgmap v1166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:21.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:21 smithi153 bash[14340]: cluster 2024-03-12T02:48:21.233771+0000 mgr.smithi153.asomxr (mgr.14195) 1399 : cluster 0 pgmap v1166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:22.777 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:48:24.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:24 smithi153 bash[14340]: cluster 2024-03-12T02:48:23.234398+0000 mgr.smithi153.asomxr (mgr.14195) 1400 : cluster 0 pgmap v1167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:24.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:24 smithi163 bash[22347]: cluster 2024-03-12T02:48:23.234398+0000 mgr.smithi153.asomxr (mgr.14195) 1400 : cluster 0 pgmap v1167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:25.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:25 smithi163 bash[22347]: cluster 2024-03-12T02:48:25.235057+0000 mgr.smithi153.asomxr (mgr.14195) 1401 : cluster 0 pgmap v1168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:25.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:25 smithi153 bash[14340]: cluster 2024-03-12T02:48:25.235057+0000 mgr.smithi153.asomxr (mgr.14195) 1401 : cluster 0 pgmap v1168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:25.925 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:48:25.925 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:48:26.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:26 smithi153 bash[14340]: audit 2024-03-12T02:48:25.909955+0000 mgr.smithi153.asomxr (mgr.14195) 1402 : audit 0 from='client.15242 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:26.732 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:48:26.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:26 smithi163 bash[22347]: audit 2024-03-12T02:48:25.909955+0000 mgr.smithi153.asomxr (mgr.14195) 1402 : audit 0 from='client.15242 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:27.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:27 smithi163 bash[22347]: cluster 2024-03-12T02:48:27.235676+0000 mgr.smithi153.asomxr (mgr.14195) 1403 : cluster 0 pgmap v1169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:27.733 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:48:27.746 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:27 smithi153 bash[14340]: cluster 2024-03-12T02:48:27.235676+0000 mgr.smithi153.asomxr (mgr.14195) 1403 : cluster 0 pgmap v1169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:29.696 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:29 smithi153 bash[14340]: cluster 2024-03-12T02:48:29.236298+0000 mgr.smithi153.asomxr (mgr.14195) 1404 : cluster 0 pgmap v1170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:29.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:29 smithi163 bash[22347]: cluster 2024-03-12T02:48:29.236298+0000 mgr.smithi153.asomxr (mgr.14195) 1404 : cluster 0 pgmap v1170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:30.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:30 smithi153 bash[14340]: audit 2024-03-12T02:48:30.436074+0000 mon.smithi153 (mon.0) 1135 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:48:30.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:30 smithi163 bash[22347]: audit 2024-03-12T02:48:30.436074+0000 mon.smithi153 (mon.0) 1135 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:48:31.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:31 smithi153 bash[14340]: cluster 2024-03-12T02:48:31.236891+0000 mgr.smithi153.asomxr (mgr.14195) 1405 : cluster 0 pgmap v1171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:31.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:31 smithi163 bash[22347]: cluster 2024-03-12T02:48:31.236891+0000 mgr.smithi153.asomxr (mgr.14195) 1405 : cluster 0 pgmap v1171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:32.485 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:48:34.522 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:34 smithi153 bash[14340]: cluster 2024-03-12T02:48:33.237605+0000 mgr.smithi153.asomxr (mgr.14195) 1406 : cluster 0 pgmap v1172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:34.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:34 smithi163 bash[22347]: cluster 2024-03-12T02:48:33.237605+0000 mgr.smithi153.asomxr (mgr.14195) 1406 : cluster 0 pgmap v1172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:35.603 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:35 smithi153 bash[14340]: cluster 2024-03-12T02:48:35.238213+0000 mgr.smithi153.asomxr (mgr.14195) 1407 : cluster 0 pgmap v1173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:35.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:35 smithi163 bash[22347]: cluster 2024-03-12T02:48:35.238213+0000 mgr.smithi153.asomxr (mgr.14195) 1407 : cluster 0 pgmap v1173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:36.056 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:48:36.057 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:48:36.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:36 smithi153 bash[14340]: audit 2024-03-12T02:48:36.045158+0000 mgr.smithi153.asomxr (mgr.14195) 1408 : audit 0 from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:36.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:36 smithi163 bash[22347]: audit 2024-03-12T02:48:36.045158+0000 mgr.smithi153.asomxr (mgr.14195) 1408 : audit 0 from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:36.835 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:48:37.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:37 smithi163 bash[22347]: cluster 2024-03-12T02:48:37.238960+0000 mgr.smithi153.asomxr (mgr.14195) 1409 : cluster 0 pgmap v1174: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:37.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:37 smithi153 bash[14340]: cluster 2024-03-12T02:48:37.238960+0000 mgr.smithi153.asomxr (mgr.14195) 1409 : cluster 0 pgmap v1174: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:37.836 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:48:39.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:39 smithi163 bash[22347]: cluster 2024-03-12T02:48:39.239574+0000 mgr.smithi153.asomxr (mgr.14195) 1410 : cluster 0 pgmap v1175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:39.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:39 smithi153 bash[14340]: cluster 2024-03-12T02:48:39.239574+0000 mgr.smithi153.asomxr (mgr.14195) 1410 : cluster 0 pgmap v1175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:41.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:41 smithi163 bash[22347]: cluster 2024-03-12T02:48:41.240202+0000 mgr.smithi153.asomxr (mgr.14195) 1411 : cluster 0 pgmap v1176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:41.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:41 smithi163 bash[22347]: audit 2024-03-12T02:48:41.323606+0000 mon.smithi153 (mon.0) 1136 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:48:41.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:41 smithi153 bash[14340]: cluster 2024-03-12T02:48:41.240202+0000 mgr.smithi153.asomxr (mgr.14195) 1411 : cluster 0 pgmap v1176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:41.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:41 smithi153 bash[14340]: audit 2024-03-12T02:48:41.323606+0000 mon.smithi153 (mon.0) 1136 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:48:42.595 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:48:42.626 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:42 smithi153 bash[14340]: audit 2024-03-12T02:48:41.897980+0000 mon.smithi153 (mon.0) 1137 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:48:42.626 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:42 smithi153 bash[14340]: audit 2024-03-12T02:48:41.899950+0000 mon.smithi153 (mon.0) 1138 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:48:42.626 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:42 smithi153 bash[14340]: cluster 2024-03-12T02:48:41.902981+0000 mgr.smithi153.asomxr (mgr.14195) 1412 : cluster 0 pgmap v1177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:42.626 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:42 smithi153 bash[14340]: cluster 2024-03-12T02:48:41.903384+0000 mgr.smithi153.asomxr (mgr.14195) 1413 : cluster 0 pgmap v1178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:42.626 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:42 smithi153 bash[14340]: audit 2024-03-12T02:48:41.909327+0000 mon.smithi153 (mon.0) 1139 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:48:42.626 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:42 smithi153 bash[14340]: audit 2024-03-12T02:48:41.913700+0000 mon.smithi153 (mon.0) 1140 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:48:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:42 smithi163 bash[22347]: audit 2024-03-12T02:48:41.897980+0000 mon.smithi153 (mon.0) 1137 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:48:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:42 smithi163 bash[22347]: audit 2024-03-12T02:48:41.899950+0000 mon.smithi153 (mon.0) 1138 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:48:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:42 smithi163 bash[22347]: cluster 2024-03-12T02:48:41.902981+0000 mgr.smithi153.asomxr (mgr.14195) 1412 : cluster 0 pgmap v1177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:42 smithi163 bash[22347]: cluster 2024-03-12T02:48:41.903384+0000 mgr.smithi153.asomxr (mgr.14195) 1413 : cluster 0 pgmap v1178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:42 smithi163 bash[22347]: audit 2024-03-12T02:48:41.909327+0000 mon.smithi153 (mon.0) 1139 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:48:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:42 smithi163 bash[22347]: audit 2024-03-12T02:48:41.913700+0000 mon.smithi153 (mon.0) 1140 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:48:45.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:45 smithi153 bash[14340]: cluster 2024-03-12T02:48:43.904022+0000 mgr.smithi153.asomxr (mgr.14195) 1414 : cluster 0 pgmap v1179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:45.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:45 smithi163 bash[22347]: cluster 2024-03-12T02:48:43.904022+0000 mgr.smithi153.asomxr (mgr.14195) 1414 : cluster 0 pgmap v1179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:45.862 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:48:45.862 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:48:46.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:46 smithi153 bash[14340]: audit 2024-03-12T02:48:45.436350+0000 mon.smithi153 (mon.0) 1141 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:48:46.723 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:48:46.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:46 smithi163 bash[22347]: audit 2024-03-12T02:48:45.436350+0000 mon.smithi153 (mon.0) 1141 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:48:47.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:47 smithi153 bash[14340]: audit 2024-03-12T02:48:45.847371+0000 mgr.smithi153.asomxr (mgr.14195) 1415 : audit 0 from='client.15250 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:47.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:47 smithi153 bash[14340]: cluster 2024-03-12T02:48:45.904607+0000 mgr.smithi153.asomxr (mgr.14195) 1416 : cluster 0 pgmap v1180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:47.724 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:48:47.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:47 smithi163 bash[22347]: audit 2024-03-12T02:48:45.847371+0000 mgr.smithi153.asomxr (mgr.14195) 1415 : audit 0 from='client.15250 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:47.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:47 smithi163 bash[22347]: cluster 2024-03-12T02:48:45.904607+0000 mgr.smithi153.asomxr (mgr.14195) 1416 : cluster 0 pgmap v1180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:49.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:49 smithi163 bash[22347]: cluster 2024-03-12T02:48:47.905210+0000 mgr.smithi153.asomxr (mgr.14195) 1417 : cluster 0 pgmap v1181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:49.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:49 smithi153 bash[14340]: cluster 2024-03-12T02:48:47.905210+0000 mgr.smithi153.asomxr (mgr.14195) 1417 : cluster 0 pgmap v1181: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:51.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:51 smithi153 bash[14340]: cluster 2024-03-12T02:48:49.905748+0000 mgr.smithi153.asomxr (mgr.14195) 1418 : cluster 0 pgmap v1182: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:51.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:51 smithi163 bash[22347]: cluster 2024-03-12T02:48:49.905748+0000 mgr.smithi153.asomxr (mgr.14195) 1418 : cluster 0 pgmap v1182: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:52.478 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:48:53.518 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:53 smithi153 bash[14340]: cluster 2024-03-12T02:48:51.906326+0000 mgr.smithi153.asomxr (mgr.14195) 1419 : cluster 0 pgmap v1183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:53.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:53 smithi163 bash[22347]: cluster 2024-03-12T02:48:51.906326+0000 mgr.smithi153.asomxr (mgr.14195) 1419 : cluster 0 pgmap v1183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:55.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:55 smithi153 bash[14340]: cluster 2024-03-12T02:48:53.906971+0000 mgr.smithi153.asomxr (mgr.14195) 1420 : cluster 0 pgmap v1184: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:55.685 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:48:55.685 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:48:55.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:55 smithi163 bash[22347]: cluster 2024-03-12T02:48:53.906971+0000 mgr.smithi153.asomxr (mgr.14195) 1420 : cluster 0 pgmap v1184: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:56.507 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:48:57.508 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:48:57.522 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:57 smithi153 bash[14340]: audit 2024-03-12T02:48:55.670292+0000 mgr.smithi153.asomxr (mgr.14195) 1421 : audit 0 from='client.15254 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:57.522 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:57 smithi153 bash[14340]: cluster 2024-03-12T02:48:55.907699+0000 mgr.smithi153.asomxr (mgr.14195) 1422 : cluster 0 pgmap v1185: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:57.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:57 smithi163 bash[22347]: audit 2024-03-12T02:48:55.670292+0000 mgr.smithi153.asomxr (mgr.14195) 1421 : audit 0 from='client.15254 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:48:57.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:57 smithi163 bash[22347]: cluster 2024-03-12T02:48:55.907699+0000 mgr.smithi153.asomxr (mgr.14195) 1422 : cluster 0 pgmap v1185: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:48:59 smithi153 bash[14340]: cluster 2024-03-12T02:48:57.908469+0000 mgr.smithi153.asomxr (mgr.14195) 1423 : cluster 0 pgmap v1186: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:48:59.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:48:59 smithi163 bash[22347]: cluster 2024-03-12T02:48:57.908469+0000 mgr.smithi153.asomxr (mgr.14195) 1423 : cluster 0 pgmap v1186: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:01 smithi153 bash[14340]: cluster 2024-03-12T02:48:59.909083+0000 mgr.smithi153.asomxr (mgr.14195) 1424 : cluster 0 pgmap v1187: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:01.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:01 smithi153 bash[14340]: audit 2024-03-12T02:49:00.436223+0000 mon.smithi153 (mon.0) 1142 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:49:01.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:01 smithi163 bash[22347]: cluster 2024-03-12T02:48:59.909083+0000 mgr.smithi153.asomxr (mgr.14195) 1424 : cluster 0 pgmap v1187: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:01.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:01 smithi163 bash[22347]: audit 2024-03-12T02:49:00.436223+0000 mon.smithi153 (mon.0) 1142 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:49:02.267 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:49:03.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:03 smithi153 bash[14340]: cluster 2024-03-12T02:49:01.909627+0000 mgr.smithi153.asomxr (mgr.14195) 1425 : cluster 0 pgmap v1188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:03.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:03 smithi163 bash[22347]: cluster 2024-03-12T02:49:01.909627+0000 mgr.smithi153.asomxr (mgr.14195) 1425 : cluster 0 pgmap v1188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:05.373 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:49:05.373 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:49:05.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:05 smithi153 bash[14340]: cluster 2024-03-12T02:49:03.910318+0000 mgr.smithi153.asomxr (mgr.14195) 1426 : cluster 0 pgmap v1189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:05.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:05 smithi163 bash[22347]: cluster 2024-03-12T02:49:03.910318+0000 mgr.smithi153.asomxr (mgr.14195) 1426 : cluster 0 pgmap v1189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:06.168 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:49:06.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:06 smithi153 bash[14340]: audit 2024-03-12T02:49:05.358767+0000 mgr.smithi153.asomxr (mgr.14195) 1427 : audit 0 from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:06 smithi153 bash[14340]: cluster 2024-03-12T02:49:05.911033+0000 mgr.smithi153.asomxr (mgr.14195) 1428 : cluster 0 pgmap v1190: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:06.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:06 smithi163 bash[22347]: audit 2024-03-12T02:49:05.358767+0000 mgr.smithi153.asomxr (mgr.14195) 1427 : audit 0 from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:06.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:06 smithi163 bash[22347]: cluster 2024-03-12T02:49:05.911033+0000 mgr.smithi153.asomxr (mgr.14195) 1428 : cluster 0 pgmap v1190: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:07.169 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:49:09.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:09 smithi153 bash[14340]: cluster 2024-03-12T02:49:07.911712+0000 mgr.smithi153.asomxr (mgr.14195) 1429 : cluster 0 pgmap v1191: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:09.624 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:09 smithi163 bash[22347]: cluster 2024-03-12T02:49:07.911712+0000 mgr.smithi153.asomxr (mgr.14195) 1429 : cluster 0 pgmap v1191: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:11.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:11 smithi153 bash[14340]: cluster 2024-03-12T02:49:09.912330+0000 mgr.smithi153.asomxr (mgr.14195) 1430 : cluster 0 pgmap v1192: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:11.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:11 smithi163 bash[22347]: cluster 2024-03-12T02:49:09.912330+0000 mgr.smithi153.asomxr (mgr.14195) 1430 : cluster 0 pgmap v1192: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:11.919 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:49:13.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:13 smithi153 bash[14340]: cluster 2024-03-12T02:49:11.912857+0000 mgr.smithi153.asomxr (mgr.14195) 1431 : cluster 0 pgmap v1193: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:13.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:13 smithi163 bash[22347]: cluster 2024-03-12T02:49:11.912857+0000 mgr.smithi153.asomxr (mgr.14195) 1431 : cluster 0 pgmap v1193: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:15.159 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:49:15.159 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:49:15.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:15 smithi153 bash[14340]: cluster 2024-03-12T02:49:13.913401+0000 mgr.smithi153.asomxr (mgr.14195) 1432 : cluster 0 pgmap v1194: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:15.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:15 smithi163 bash[22347]: cluster 2024-03-12T02:49:13.913401+0000 mgr.smithi153.asomxr (mgr.14195) 1432 : cluster 0 pgmap v1194: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:15.891 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:49:16.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:16 smithi153 bash[14340]: audit 2024-03-12T02:49:15.143544+0000 mgr.smithi153.asomxr (mgr.14195) 1433 : audit 0 from='client.15262 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:16 smithi153 bash[14340]: audit 2024-03-12T02:49:15.437427+0000 mon.smithi153 (mon.0) 1143 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:49:16.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:16 smithi163 bash[22347]: audit 2024-03-12T02:49:15.143544+0000 mgr.smithi153.asomxr (mgr.14195) 1433 : audit 0 from='client.15262 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:16 smithi163 bash[22347]: audit 2024-03-12T02:49:15.437427+0000 mon.smithi153 (mon.0) 1143 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:49:16.892 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:49:17.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:17 smithi153 bash[14340]: cluster 2024-03-12T02:49:15.913990+0000 mgr.smithi153.asomxr (mgr.14195) 1434 : cluster 0 pgmap v1195: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:17.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:17 smithi163 bash[22347]: cluster 2024-03-12T02:49:15.913990+0000 mgr.smithi153.asomxr (mgr.14195) 1434 : cluster 0 pgmap v1195: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:19.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:19 smithi153 bash[14340]: cluster 2024-03-12T02:49:17.914593+0000 mgr.smithi153.asomxr (mgr.14195) 1435 : cluster 0 pgmap v1196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:19.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:19 smithi163 bash[22347]: cluster 2024-03-12T02:49:17.914593+0000 mgr.smithi153.asomxr (mgr.14195) 1435 : cluster 0 pgmap v1196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:21.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:21 smithi153 bash[14340]: cluster 2024-03-12T02:49:19.915298+0000 mgr.smithi153.asomxr (mgr.14195) 1436 : cluster 0 pgmap v1197: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:21.647 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:49:21.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:21 smithi163 bash[22347]: cluster 2024-03-12T02:49:19.915298+0000 mgr.smithi153.asomxr (mgr.14195) 1436 : cluster 0 pgmap v1197: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:23.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:23 smithi153 bash[14340]: cluster 2024-03-12T02:49:21.915863+0000 mgr.smithi153.asomxr (mgr.14195) 1437 : cluster 0 pgmap v1198: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:23.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:23 smithi163 bash[22347]: cluster 2024-03-12T02:49:21.915863+0000 mgr.smithi153.asomxr (mgr.14195) 1437 : cluster 0 pgmap v1198: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:24.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:24 smithi163 bash[22347]: cluster 2024-03-12T02:49:23.916386+0000 mgr.smithi153.asomxr (mgr.14195) 1438 : cluster 0 pgmap v1199: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:24.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:24 smithi153 bash[14340]: cluster 2024-03-12T02:49:23.916386+0000 mgr.smithi153.asomxr (mgr.14195) 1438 : cluster 0 pgmap v1199: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:24.962 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:49:24.963 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:49:25.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:25 smithi153 bash[14340]: audit 2024-03-12T02:49:24.947360+0000 mgr.smithi153.asomxr (mgr.14195) 1439 : audit 0 from='client.15266 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:25.730 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:49:25.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:25 smithi163 bash[22347]: audit 2024-03-12T02:49:24.947360+0000 mgr.smithi153.asomxr (mgr.14195) 1439 : audit 0 from='client.15266 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:26.731 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:49:26.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:26 smithi163 bash[22347]: cluster 2024-03-12T02:49:25.917022+0000 mgr.smithi153.asomxr (mgr.14195) 1440 : cluster 0 pgmap v1200: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:26.745 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:26 smithi153 bash[14340]: cluster 2024-03-12T02:49:25.917022+0000 mgr.smithi153.asomxr (mgr.14195) 1440 : cluster 0 pgmap v1200: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:29.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:29 smithi153 bash[14340]: cluster 2024-03-12T02:49:27.917508+0000 mgr.smithi153.asomxr (mgr.14195) 1441 : cluster 0 pgmap v1201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:29.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:29 smithi163 bash[22347]: cluster 2024-03-12T02:49:27.917508+0000 mgr.smithi153.asomxr (mgr.14195) 1441 : cluster 0 pgmap v1201: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:31.485 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:49:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:31 smithi153 bash[14340]: cluster 2024-03-12T02:49:29.918092+0000 mgr.smithi153.asomxr (mgr.14195) 1442 : cluster 0 pgmap v1202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:31 smithi153 bash[14340]: audit 2024-03-12T02:49:30.437796+0000 mon.smithi153 (mon.0) 1144 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:49:31.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:31 smithi163 bash[22347]: cluster 2024-03-12T02:49:29.918092+0000 mgr.smithi153.asomxr (mgr.14195) 1442 : cluster 0 pgmap v1202: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:31.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:31 smithi163 bash[22347]: audit 2024-03-12T02:49:30.437796+0000 mon.smithi153 (mon.0) 1144 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:49:33.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:33 smithi153 bash[14340]: cluster 2024-03-12T02:49:31.918725+0000 mgr.smithi153.asomxr (mgr.14195) 1443 : cluster 0 pgmap v1203: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:33.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:33 smithi163 bash[22347]: cluster 2024-03-12T02:49:31.918725+0000 mgr.smithi153.asomxr (mgr.14195) 1443 : cluster 0 pgmap v1203: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:34.727 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:49:34.728 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:49:35.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:35 smithi163 bash[22347]: cluster 2024-03-12T02:49:33.919430+0000 mgr.smithi153.asomxr (mgr.14195) 1444 : cluster 0 pgmap v1204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:35.492 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:49:35.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:35 smithi153 bash[14340]: cluster 2024-03-12T02:49:33.919430+0000 mgr.smithi153.asomxr (mgr.14195) 1444 : cluster 0 pgmap v1204: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:36.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:36 smithi153 bash[14340]: audit 2024-03-12T02:49:34.716930+0000 mgr.smithi153.asomxr (mgr.14195) 1445 : audit 0 from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:36.493 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:49:36.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:36 smithi163 bash[22347]: audit 2024-03-12T02:49:34.716930+0000 mgr.smithi153.asomxr (mgr.14195) 1445 : audit 0 from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:37.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:37 smithi153 bash[14340]: cluster 2024-03-12T02:49:35.920082+0000 mgr.smithi153.asomxr (mgr.14195) 1446 : cluster 0 pgmap v1205: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:37.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:37 smithi163 bash[22347]: cluster 2024-03-12T02:49:35.920082+0000 mgr.smithi153.asomxr (mgr.14195) 1446 : cluster 0 pgmap v1205: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:39.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:39 smithi153 bash[14340]: cluster 2024-03-12T02:49:37.920806+0000 mgr.smithi153.asomxr (mgr.14195) 1447 : cluster 0 pgmap v1206: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:39.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:39 smithi163 bash[22347]: cluster 2024-03-12T02:49:37.920806+0000 mgr.smithi153.asomxr (mgr.14195) 1447 : cluster 0 pgmap v1206: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:41.242 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:49:41.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:41 smithi153 bash[14340]: cluster 2024-03-12T02:49:39.921592+0000 mgr.smithi153.asomxr (mgr.14195) 1448 : cluster 0 pgmap v1207: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:41.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:41 smithi163 bash[22347]: cluster 2024-03-12T02:49:39.921592+0000 mgr.smithi153.asomxr (mgr.14195) 1448 : cluster 0 pgmap v1207: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:42.498 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:42 smithi153 bash[14340]: audit 2024-03-12T02:49:42.008601+0000 mon.smithi153 (mon.0) 1145 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:49:42.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:42 smithi163 bash[22347]: audit 2024-03-12T02:49:42.008601+0000 mon.smithi153 (mon.0) 1145 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:49:43.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:43 smithi153 bash[14340]: cluster 2024-03-12T02:49:41.922223+0000 mgr.smithi153.asomxr (mgr.14195) 1449 : cluster 0 pgmap v1208: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:43.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:43 smithi153 bash[14340]: audit 2024-03-12T02:49:42.595967+0000 mon.smithi153 (mon.0) 1146 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:49:43.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:43 smithi153 bash[14340]: audit 2024-03-12T02:49:42.596908+0000 mon.smithi153 (mon.0) 1147 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:49:43.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:43 smithi153 bash[14340]: cluster 2024-03-12T02:49:42.598636+0000 mgr.smithi153.asomxr (mgr.14195) 1450 : cluster 0 pgmap v1209: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:43.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:43 smithi153 bash[14340]: cluster 2024-03-12T02:49:42.599018+0000 mgr.smithi153.asomxr (mgr.14195) 1451 : cluster 0 pgmap v1210: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:43.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:43 smithi153 bash[14340]: audit 2024-03-12T02:49:42.606432+0000 mon.smithi153 (mon.0) 1148 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:49:43.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:43 smithi153 bash[14340]: audit 2024-03-12T02:49:42.609762+0000 mon.smithi153 (mon.0) 1149 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:49:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:43 smithi163 bash[22347]: cluster 2024-03-12T02:49:41.922223+0000 mgr.smithi153.asomxr (mgr.14195) 1449 : cluster 0 pgmap v1208: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:43 smithi163 bash[22347]: audit 2024-03-12T02:49:42.595967+0000 mon.smithi153 (mon.0) 1146 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:49:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:43 smithi163 bash[22347]: audit 2024-03-12T02:49:42.596908+0000 mon.smithi153 (mon.0) 1147 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:49:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:43 smithi163 bash[22347]: cluster 2024-03-12T02:49:42.598636+0000 mgr.smithi153.asomxr (mgr.14195) 1450 : cluster 0 pgmap v1209: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:43 smithi163 bash[22347]: cluster 2024-03-12T02:49:42.599018+0000 mgr.smithi153.asomxr (mgr.14195) 1451 : cluster 0 pgmap v1210: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:43 smithi163 bash[22347]: audit 2024-03-12T02:49:42.606432+0000 mon.smithi153 (mon.0) 1148 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:49:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:43 smithi163 bash[22347]: audit 2024-03-12T02:49:42.609762+0000 mon.smithi153 (mon.0) 1149 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:49:44.621 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:49:44.621 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:49:45.431 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:49:45.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:45 smithi153 bash[14340]: cluster 2024-03-12T02:49:44.599430+0000 mgr.smithi153.asomxr (mgr.14195) 1452 : cluster 0 pgmap v1211: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:45.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:45 smithi153 bash[14340]: audit 2024-03-12T02:49:44.606282+0000 mgr.smithi153.asomxr (mgr.14195) 1453 : audit 0 from='client.15274 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:45.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:45 smithi153 bash[14340]: audit 2024-03-12T02:49:45.437900+0000 mon.smithi153 (mon.0) 1150 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:49:45.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:45 smithi163 bash[22347]: cluster 2024-03-12T02:49:44.599430+0000 mgr.smithi153.asomxr (mgr.14195) 1452 : cluster 0 pgmap v1211: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:45.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:45 smithi163 bash[22347]: audit 2024-03-12T02:49:44.606282+0000 mgr.smithi153.asomxr (mgr.14195) 1453 : audit 0 from='client.15274 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:45.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:45 smithi163 bash[22347]: audit 2024-03-12T02:49:45.437900+0000 mon.smithi153 (mon.0) 1150 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:49:46.432 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:49:47.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:47 smithi163 bash[22347]: cluster 2024-03-12T02:49:46.600056+0000 mgr.smithi153.asomxr (mgr.14195) 1454 : cluster 0 pgmap v1212: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:48.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:47 smithi153 bash[14340]: cluster 2024-03-12T02:49:46.600056+0000 mgr.smithi153.asomxr (mgr.14195) 1454 : cluster 0 pgmap v1212: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:50.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:50 smithi153 bash[14340]: cluster 2024-03-12T02:49:48.600730+0000 mgr.smithi153.asomxr (mgr.14195) 1455 : cluster 0 pgmap v1213: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:50.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:50 smithi163 bash[22347]: cluster 2024-03-12T02:49:48.600730+0000 mgr.smithi153.asomxr (mgr.14195) 1455 : cluster 0 pgmap v1213: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:51.187 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:49:52.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:52 smithi153 bash[14340]: cluster 2024-03-12T02:49:50.601430+0000 mgr.smithi153.asomxr (mgr.14195) 1456 : cluster 0 pgmap v1214: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:52.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:52 smithi163 bash[22347]: cluster 2024-03-12T02:49:50.601430+0000 mgr.smithi153.asomxr (mgr.14195) 1456 : cluster 0 pgmap v1214: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:54.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:54 smithi153 bash[14340]: cluster 2024-03-12T02:49:52.602045+0000 mgr.smithi153.asomxr (mgr.14195) 1457 : cluster 0 pgmap v1215: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:54.622 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:49:54.622 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:49:54.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:54 smithi163 bash[22347]: cluster 2024-03-12T02:49:52.602045+0000 mgr.smithi153.asomxr (mgr.14195) 1457 : cluster 0 pgmap v1215: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:55.367 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:49:56.368 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:49:56.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:56 smithi153 bash[14340]: cluster 2024-03-12T02:49:54.602630+0000 mgr.smithi153.asomxr (mgr.14195) 1458 : cluster 0 pgmap v1216: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:56.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:56 smithi153 bash[14340]: audit 2024-03-12T02:49:54.610398+0000 mgr.smithi153.asomxr (mgr.14195) 1459 : audit 0 from='client.15278 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:56.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:56 smithi163 bash[22347]: cluster 2024-03-12T02:49:54.602630+0000 mgr.smithi153.asomxr (mgr.14195) 1458 : cluster 0 pgmap v1216: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:56.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:56 smithi163 bash[22347]: audit 2024-03-12T02:49:54.610398+0000 mgr.smithi153.asomxr (mgr.14195) 1459 : audit 0 from='client.15278 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:49:58.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:49:58 smithi153 bash[14340]: cluster 2024-03-12T02:49:56.603250+0000 mgr.smithi153.asomxr (mgr.14195) 1460 : cluster 0 pgmap v1217: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:49:58.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:49:58 smithi163 bash[22347]: cluster 2024-03-12T02:49:56.603250+0000 mgr.smithi153.asomxr (mgr.14195) 1460 : cluster 0 pgmap v1217: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:00.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:00 smithi153 bash[14340]: cluster 2024-03-12T02:49:58.603886+0000 mgr.smithi153.asomxr (mgr.14195) 1461 : cluster 0 pgmap v1218: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:00.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:00 smithi153 bash[14340]: cluster 2024-03-12T02:50:00.000120+0000 mon.smithi153 (mon.0) 1151 : cluster 3 overall HEALTH_WARN 2 failed cephadm daemon(s) 2024-03-12T02:50:00.655 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:00 smithi163 bash[22347]: cluster 2024-03-12T02:49:58.603886+0000 mgr.smithi153.asomxr (mgr.14195) 1461 : cluster 0 pgmap v1218: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:00.655 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:00 smithi163 bash[22347]: cluster 2024-03-12T02:50:00.000120+0000 mon.smithi153 (mon.0) 1151 : cluster 3 overall HEALTH_WARN 2 failed cephadm daemon(s) 2024-03-12T02:50:01.119 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:50:01.519 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:01 smithi153 bash[14340]: audit 2024-03-12T02:50:00.438298+0000 mon.smithi153 (mon.0) 1152 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:50:01.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:01 smithi163 bash[22347]: audit 2024-03-12T02:50:00.438298+0000 mon.smithi153 (mon.0) 1152 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:50:02.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:02 smithi153 bash[14340]: cluster 2024-03-12T02:50:00.604409+0000 mgr.smithi153.asomxr (mgr.14195) 1462 : cluster 0 pgmap v1219: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:02.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:02 smithi163 bash[22347]: cluster 2024-03-12T02:50:00.604409+0000 mgr.smithi153.asomxr (mgr.14195) 1462 : cluster 0 pgmap v1219: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:04.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:04 smithi153 bash[14340]: cluster 2024-03-12T02:50:02.605018+0000 mgr.smithi153.asomxr (mgr.14195) 1463 : cluster 0 pgmap v1220: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:04.673 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:50:04.674 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:50:04.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:04 smithi163 bash[22347]: cluster 2024-03-12T02:50:02.605018+0000 mgr.smithi153.asomxr (mgr.14195) 1463 : cluster 0 pgmap v1220: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:05.479 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:50:06.480 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:50:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:06 smithi153 bash[14340]: cluster 2024-03-12T02:50:04.605648+0000 mgr.smithi153.asomxr (mgr.14195) 1464 : cluster 0 pgmap v1221: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:06 smithi153 bash[14340]: audit 2024-03-12T02:50:04.663162+0000 mgr.smithi153.asomxr (mgr.14195) 1465 : audit 0 from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:06.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:06 smithi163 bash[22347]: cluster 2024-03-12T02:50:04.605648+0000 mgr.smithi153.asomxr (mgr.14195) 1464 : cluster 0 pgmap v1221: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:06.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:06 smithi163 bash[22347]: audit 2024-03-12T02:50:04.663162+0000 mgr.smithi153.asomxr (mgr.14195) 1465 : audit 0 from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:08.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:08 smithi153 bash[14340]: cluster 2024-03-12T02:50:06.606234+0000 mgr.smithi153.asomxr (mgr.14195) 1466 : cluster 0 pgmap v1222: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:08.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:08 smithi163 bash[22347]: cluster 2024-03-12T02:50:06.606234+0000 mgr.smithi153.asomxr (mgr.14195) 1466 : cluster 0 pgmap v1222: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:09.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:09 smithi153 bash[14340]: cluster 2024-03-12T02:50:08.606753+0000 mgr.smithi153.asomxr (mgr.14195) 1467 : cluster 0 pgmap v1223: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:09.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:09 smithi163 bash[22347]: cluster 2024-03-12T02:50:08.606753+0000 mgr.smithi153.asomxr (mgr.14195) 1467 : cluster 0 pgmap v1223: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:11.235 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:50:11.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:11 smithi163 bash[22347]: cluster 2024-03-12T02:50:10.607378+0000 mgr.smithi153.asomxr (mgr.14195) 1468 : cluster 0 pgmap v1224: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:11 smithi153 bash[14340]: cluster 2024-03-12T02:50:10.607378+0000 mgr.smithi153.asomxr (mgr.14195) 1468 : cluster 0 pgmap v1224: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:14.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:14 smithi153 bash[14340]: cluster 2024-03-12T02:50:12.607786+0000 mgr.smithi153.asomxr (mgr.14195) 1469 : cluster 0 pgmap v1225: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:14.703 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:50:14.703 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:50:14.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:14 smithi163 bash[22347]: cluster 2024-03-12T02:50:12.607786+0000 mgr.smithi153.asomxr (mgr.14195) 1469 : cluster 0 pgmap v1225: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:15.512 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:50:16.514 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:50:16.525 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:16 smithi153 bash[14340]: cluster 2024-03-12T02:50:14.608325+0000 mgr.smithi153.asomxr (mgr.14195) 1470 : cluster 0 pgmap v1226: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:16.525 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:16 smithi153 bash[14340]: audit 2024-03-12T02:50:14.688205+0000 mgr.smithi153.asomxr (mgr.14195) 1471 : audit 0 from='client.15286 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:16.525 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:16 smithi153 bash[14340]: audit 2024-03-12T02:50:15.438875+0000 mon.smithi153 (mon.0) 1153 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:50:16.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:16 smithi163 bash[22347]: cluster 2024-03-12T02:50:14.608325+0000 mgr.smithi153.asomxr (mgr.14195) 1470 : cluster 0 pgmap v1226: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:16 smithi163 bash[22347]: audit 2024-03-12T02:50:14.688205+0000 mgr.smithi153.asomxr (mgr.14195) 1471 : audit 0 from='client.15286 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:16 smithi163 bash[22347]: audit 2024-03-12T02:50:15.438875+0000 mon.smithi153 (mon.0) 1153 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:50:18.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:18 smithi153 bash[14340]: cluster 2024-03-12T02:50:16.608937+0000 mgr.smithi153.asomxr (mgr.14195) 1472 : cluster 0 pgmap v1227: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:18.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:18 smithi163 bash[22347]: cluster 2024-03-12T02:50:16.608937+0000 mgr.smithi153.asomxr (mgr.14195) 1472 : cluster 0 pgmap v1227: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:20.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:20 smithi153 bash[14340]: cluster 2024-03-12T02:50:18.609567+0000 mgr.smithi153.asomxr (mgr.14195) 1473 : cluster 0 pgmap v1228: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:20.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:20 smithi163 bash[22347]: cluster 2024-03-12T02:50:18.609567+0000 mgr.smithi153.asomxr (mgr.14195) 1473 : cluster 0 pgmap v1228: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:21.268 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:50:22.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:22 smithi153 bash[14340]: cluster 2024-03-12T02:50:20.610121+0000 mgr.smithi153.asomxr (mgr.14195) 1474 : cluster 0 pgmap v1229: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:22.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:22 smithi163 bash[22347]: cluster 2024-03-12T02:50:20.610121+0000 mgr.smithi153.asomxr (mgr.14195) 1474 : cluster 0 pgmap v1229: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:24.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:24 smithi153 bash[14340]: cluster 2024-03-12T02:50:22.610792+0000 mgr.smithi153.asomxr (mgr.14195) 1475 : cluster 0 pgmap v1230: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:24.693 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:50:24.693 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:50:24.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:24 smithi163 bash[22347]: cluster 2024-03-12T02:50:22.610792+0000 mgr.smithi153.asomxr (mgr.14195) 1475 : cluster 0 pgmap v1230: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:25.579 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:50:26.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:26 smithi163 bash[22347]: cluster 2024-03-12T02:50:24.611265+0000 mgr.smithi153.asomxr (mgr.14195) 1476 : cluster 0 pgmap v1231: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:26 smithi163 bash[22347]: audit 2024-03-12T02:50:24.677655+0000 mgr.smithi153.asomxr (mgr.14195) 1477 : audit 0 from='client.15290 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:26.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:26 smithi153 bash[14340]: cluster 2024-03-12T02:50:24.611265+0000 mgr.smithi153.asomxr (mgr.14195) 1476 : cluster 0 pgmap v1231: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:26 smithi153 bash[14340]: audit 2024-03-12T02:50:24.677655+0000 mgr.smithi153.asomxr (mgr.14195) 1477 : audit 0 from='client.15290 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:26.580 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:50:28.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:28 smithi153 bash[14340]: cluster 2024-03-12T02:50:26.611838+0000 mgr.smithi153.asomxr (mgr.14195) 1478 : cluster 0 pgmap v1232: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:28.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:28 smithi163 bash[22347]: cluster 2024-03-12T02:50:26.611838+0000 mgr.smithi153.asomxr (mgr.14195) 1478 : cluster 0 pgmap v1232: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:30.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:30 smithi153 bash[14340]: cluster 2024-03-12T02:50:28.612436+0000 mgr.smithi153.asomxr (mgr.14195) 1479 : cluster 0 pgmap v1233: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:30.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:30 smithi163 bash[22347]: cluster 2024-03-12T02:50:28.612436+0000 mgr.smithi153.asomxr (mgr.14195) 1479 : cluster 0 pgmap v1233: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:31.329 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:50:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:31 smithi153 bash[14340]: audit 2024-03-12T02:50:30.439176+0000 mon.smithi153 (mon.0) 1154 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:50:31.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:31 smithi163 bash[22347]: audit 2024-03-12T02:50:30.439176+0000 mon.smithi153 (mon.0) 1154 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:50:32.365 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:32 smithi153 bash[14340]: cluster 2024-03-12T02:50:30.613049+0000 mgr.smithi153.asomxr (mgr.14195) 1480 : cluster 0 pgmap v1234: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:32.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:32 smithi163 bash[22347]: cluster 2024-03-12T02:50:30.613049+0000 mgr.smithi153.asomxr (mgr.14195) 1480 : cluster 0 pgmap v1234: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:34.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:34 smithi153 bash[14340]: cluster 2024-03-12T02:50:32.613650+0000 mgr.smithi153.asomxr (mgr.14195) 1481 : cluster 0 pgmap v1235: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:34.665 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:50:34.665 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:50:34.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:34 smithi163 bash[22347]: cluster 2024-03-12T02:50:32.613650+0000 mgr.smithi153.asomxr (mgr.14195) 1481 : cluster 0 pgmap v1235: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:35.449 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:50:36.450 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:50:36.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:36 smithi153 bash[14340]: cluster 2024-03-12T02:50:34.614353+0000 mgr.smithi153.asomxr (mgr.14195) 1482 : cluster 0 pgmap v1236: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:36.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:36 smithi153 bash[14340]: audit 2024-03-12T02:50:34.650261+0000 mgr.smithi153.asomxr (mgr.14195) 1483 : audit 0 from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:36.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:36 smithi163 bash[22347]: cluster 2024-03-12T02:50:34.614353+0000 mgr.smithi153.asomxr (mgr.14195) 1482 : cluster 0 pgmap v1236: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:36.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:36 smithi163 bash[22347]: audit 2024-03-12T02:50:34.650261+0000 mgr.smithi153.asomxr (mgr.14195) 1483 : audit 0 from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:38.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:38 smithi153 bash[14340]: cluster 2024-03-12T02:50:36.614922+0000 mgr.smithi153.asomxr (mgr.14195) 1484 : cluster 0 pgmap v1237: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:38.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:38 smithi163 bash[22347]: cluster 2024-03-12T02:50:36.614922+0000 mgr.smithi153.asomxr (mgr.14195) 1484 : cluster 0 pgmap v1237: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:40.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:40 smithi153 bash[14340]: cluster 2024-03-12T02:50:38.615556+0000 mgr.smithi153.asomxr (mgr.14195) 1485 : cluster 0 pgmap v1238: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:40.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:40 smithi163 bash[22347]: cluster 2024-03-12T02:50:38.615556+0000 mgr.smithi153.asomxr (mgr.14195) 1485 : cluster 0 pgmap v1238: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:41.203 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:50:42.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:42 smithi153 bash[14340]: cluster 2024-03-12T02:50:40.616204+0000 mgr.smithi153.asomxr (mgr.14195) 1486 : cluster 0 pgmap v1239: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:42.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:42 smithi163 bash[22347]: cluster 2024-03-12T02:50:40.616204+0000 mgr.smithi153.asomxr (mgr.14195) 1486 : cluster 0 pgmap v1239: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:43.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:43 smithi153 bash[14340]: audit 2024-03-12T02:50:42.690373+0000 mon.smithi153 (mon.0) 1155 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:50:43.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:43 smithi153 bash[14340]: audit 2024-03-12T02:50:43.224324+0000 mon.smithi153 (mon.0) 1156 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:50:43.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:43 smithi153 bash[14340]: audit 2024-03-12T02:50:43.225722+0000 mon.smithi153 (mon.0) 1157 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:50:43.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:43 smithi153 bash[14340]: audit 2024-03-12T02:50:43.234664+0000 mon.smithi153 (mon.0) 1158 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:50:43.575 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:43 smithi153 bash[14340]: audit 2024-03-12T02:50:43.240433+0000 mon.smithi153 (mon.0) 1159 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:50:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:43 smithi163 bash[22347]: audit 2024-03-12T02:50:42.690373+0000 mon.smithi153 (mon.0) 1155 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:50:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:43 smithi163 bash[22347]: audit 2024-03-12T02:50:43.224324+0000 mon.smithi153 (mon.0) 1156 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:50:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:43 smithi163 bash[22347]: audit 2024-03-12T02:50:43.225722+0000 mon.smithi153 (mon.0) 1157 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:50:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:43 smithi163 bash[22347]: audit 2024-03-12T02:50:43.234664+0000 mon.smithi153 (mon.0) 1158 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:50:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:43 smithi163 bash[22347]: audit 2024-03-12T02:50:43.240433+0000 mon.smithi153 (mon.0) 1159 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:50:44.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:44 smithi153 bash[14340]: cluster 2024-03-12T02:50:42.616754+0000 mgr.smithi153.asomxr (mgr.14195) 1487 : cluster 0 pgmap v1240: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:44.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:44 smithi153 bash[14340]: cluster 2024-03-12T02:50:43.228192+0000 mgr.smithi153.asomxr (mgr.14195) 1488 : cluster 0 pgmap v1241: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:44.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:44 smithi153 bash[14340]: cluster 2024-03-12T02:50:43.228532+0000 mgr.smithi153.asomxr (mgr.14195) 1489 : cluster 0 pgmap v1242: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:44.728 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:50:44.728 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:50:44.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:44 smithi163 bash[22347]: cluster 2024-03-12T02:50:42.616754+0000 mgr.smithi153.asomxr (mgr.14195) 1487 : cluster 0 pgmap v1240: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:44.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:44 smithi163 bash[22347]: cluster 2024-03-12T02:50:43.228192+0000 mgr.smithi153.asomxr (mgr.14195) 1488 : cluster 0 pgmap v1241: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:44.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:44 smithi163 bash[22347]: cluster 2024-03-12T02:50:43.228532+0000 mgr.smithi153.asomxr (mgr.14195) 1489 : cluster 0 pgmap v1242: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:45.555 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:50:46.557 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:50:46.568 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:46 smithi153 bash[14340]: audit 2024-03-12T02:50:44.712172+0000 mgr.smithi153.asomxr (mgr.14195) 1490 : audit 0 from='client.15298 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:46.568 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:46 smithi153 bash[14340]: cluster 2024-03-12T02:50:45.229110+0000 mgr.smithi153.asomxr (mgr.14195) 1491 : cluster 0 pgmap v1243: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:46.569 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:46 smithi153 bash[14340]: audit 2024-03-12T02:50:45.439577+0000 mon.smithi153 (mon.0) 1160 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:50:46.614 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:46 smithi163 bash[22347]: audit 2024-03-12T02:50:44.712172+0000 mgr.smithi153.asomxr (mgr.14195) 1490 : audit 0 from='client.15298 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:46.614 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:46 smithi163 bash[22347]: cluster 2024-03-12T02:50:45.229110+0000 mgr.smithi153.asomxr (mgr.14195) 1491 : cluster 0 pgmap v1243: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:46.614 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:46 smithi163 bash[22347]: audit 2024-03-12T02:50:45.439577+0000 mon.smithi153 (mon.0) 1160 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:50:48.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:48 smithi153 bash[14340]: cluster 2024-03-12T02:50:47.229696+0000 mgr.smithi153.asomxr (mgr.14195) 1492 : cluster 0 pgmap v1244: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:48.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:48 smithi163 bash[22347]: cluster 2024-03-12T02:50:47.229696+0000 mgr.smithi153.asomxr (mgr.14195) 1492 : cluster 0 pgmap v1244: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:49.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:49 smithi163 bash[22347]: cluster 2024-03-12T02:50:49.230335+0000 mgr.smithi153.asomxr (mgr.14195) 1493 : cluster 0 pgmap v1245: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:49.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:49 smithi153 bash[14340]: cluster 2024-03-12T02:50:49.230335+0000 mgr.smithi153.asomxr (mgr.14195) 1493 : cluster 0 pgmap v1245: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:51.308 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:50:52.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:52 smithi153 bash[14340]: cluster 2024-03-12T02:50:51.230979+0000 mgr.smithi153.asomxr (mgr.14195) 1494 : cluster 0 pgmap v1246: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:52.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:52 smithi163 bash[22347]: cluster 2024-03-12T02:50:51.230979+0000 mgr.smithi153.asomxr (mgr.14195) 1494 : cluster 0 pgmap v1246: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:54.553 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:50:54.553 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:50:54.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:54 smithi153 bash[14340]: cluster 2024-03-12T02:50:53.231392+0000 mgr.smithi153.asomxr (mgr.14195) 1495 : cluster 0 pgmap v1247: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:54.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:54 smithi163 bash[22347]: cluster 2024-03-12T02:50:53.231392+0000 mgr.smithi153.asomxr (mgr.14195) 1495 : cluster 0 pgmap v1247: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:55.376 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:50:56.377 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:50:56.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:56 smithi153 bash[14340]: audit 2024-03-12T02:50:54.537965+0000 mgr.smithi153.asomxr (mgr.14195) 1496 : audit 0 from='client.15302 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:56.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:56 smithi153 bash[14340]: cluster 2024-03-12T02:50:55.231901+0000 mgr.smithi153.asomxr (mgr.14195) 1497 : cluster 0 pgmap v1248: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:56.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:56 smithi163 bash[22347]: audit 2024-03-12T02:50:54.537965+0000 mgr.smithi153.asomxr (mgr.14195) 1496 : audit 0 from='client.15302 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:50:56.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:56 smithi163 bash[22347]: cluster 2024-03-12T02:50:55.231901+0000 mgr.smithi153.asomxr (mgr.14195) 1497 : cluster 0 pgmap v1248: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:58.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:50:58 smithi153 bash[14340]: cluster 2024-03-12T02:50:57.232475+0000 mgr.smithi153.asomxr (mgr.14195) 1498 : cluster 0 pgmap v1249: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:50:58.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:50:58 smithi163 bash[22347]: cluster 2024-03-12T02:50:57.232475+0000 mgr.smithi153.asomxr (mgr.14195) 1498 : cluster 0 pgmap v1249: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:00.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:00 smithi153 bash[14340]: cluster 2024-03-12T02:50:59.233110+0000 mgr.smithi153.asomxr (mgr.14195) 1499 : cluster 0 pgmap v1250: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:00.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:00 smithi163 bash[22347]: cluster 2024-03-12T02:50:59.233110+0000 mgr.smithi153.asomxr (mgr.14195) 1499 : cluster 0 pgmap v1250: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:01.131 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:51:01.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:01 smithi153 bash[14340]: audit 2024-03-12T02:51:00.439971+0000 mon.smithi153 (mon.0) 1161 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:51:01.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:01 smithi163 bash[22347]: audit 2024-03-12T02:51:00.439971+0000 mon.smithi153 (mon.0) 1161 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:51:02.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:02 smithi153 bash[14340]: cluster 2024-03-12T02:51:01.233730+0000 mgr.smithi153.asomxr (mgr.14195) 1500 : cluster 0 pgmap v1251: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:02.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:02 smithi163 bash[22347]: cluster 2024-03-12T02:51:01.233730+0000 mgr.smithi153.asomxr (mgr.14195) 1500 : cluster 0 pgmap v1251: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:04.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:04 smithi153 bash[14340]: cluster 2024-03-12T02:51:03.234364+0000 mgr.smithi153.asomxr (mgr.14195) 1501 : cluster 0 pgmap v1252: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:04.507 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:51:04.507 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:51:04.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:04 smithi163 bash[22347]: cluster 2024-03-12T02:51:03.234364+0000 mgr.smithi153.asomxr (mgr.14195) 1501 : cluster 0 pgmap v1252: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:05.353 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:51:06.354 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:51:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:06 smithi153 bash[14340]: audit 2024-03-12T02:51:04.495686+0000 mgr.smithi153.asomxr (mgr.14195) 1502 : audit 0 from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:06 smithi153 bash[14340]: cluster 2024-03-12T02:51:05.235014+0000 mgr.smithi153.asomxr (mgr.14195) 1503 : cluster 0 pgmap v1253: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:06.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:06 smithi163 bash[22347]: audit 2024-03-12T02:51:04.495686+0000 mgr.smithi153.asomxr (mgr.14195) 1502 : audit 0 from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:06.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:06 smithi163 bash[22347]: cluster 2024-03-12T02:51:05.235014+0000 mgr.smithi153.asomxr (mgr.14195) 1503 : cluster 0 pgmap v1253: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:08.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:08 smithi153 bash[14340]: cluster 2024-03-12T02:51:07.235619+0000 mgr.smithi153.asomxr (mgr.14195) 1504 : cluster 0 pgmap v1254: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:08.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:08 smithi163 bash[22347]: cluster 2024-03-12T02:51:07.235619+0000 mgr.smithi153.asomxr (mgr.14195) 1504 : cluster 0 pgmap v1254: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:10.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:10 smithi153 bash[14340]: cluster 2024-03-12T02:51:09.236272+0000 mgr.smithi153.asomxr (mgr.14195) 1505 : cluster 0 pgmap v1255: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:10.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:10 smithi163 bash[22347]: cluster 2024-03-12T02:51:09.236272+0000 mgr.smithi153.asomxr (mgr.14195) 1505 : cluster 0 pgmap v1255: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:11.107 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:51:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:12 smithi163 bash[22347]: cluster 2024-03-12T02:51:11.236931+0000 mgr.smithi153.asomxr (mgr.14195) 1506 : cluster 0 pgmap v1256: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:12.491 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:12 smithi153 bash[14340]: cluster 2024-03-12T02:51:11.236931+0000 mgr.smithi153.asomxr (mgr.14195) 1506 : cluster 0 pgmap v1256: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:14.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:14 smithi153 bash[14340]: cluster 2024-03-12T02:51:13.237607+0000 mgr.smithi153.asomxr (mgr.14195) 1507 : cluster 0 pgmap v1257: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:14.652 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:51:14.652 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:51:14.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:14 smithi163 bash[22347]: cluster 2024-03-12T02:51:13.237607+0000 mgr.smithi153.asomxr (mgr.14195) 1507 : cluster 0 pgmap v1257: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:15.490 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:51:16.491 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:51:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:16 smithi153 bash[14340]: audit 2024-03-12T02:51:14.637871+0000 mgr.smithi153.asomxr (mgr.14195) 1508 : audit 0 from='client.15310 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:16 smithi153 bash[14340]: cluster 2024-03-12T02:51:15.238201+0000 mgr.smithi153.asomxr (mgr.14195) 1509 : cluster 0 pgmap v1258: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:16.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:16 smithi153 bash[14340]: audit 2024-03-12T02:51:15.439703+0000 mon.smithi153 (mon.0) 1162 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:51:16.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:16 smithi163 bash[22347]: audit 2024-03-12T02:51:14.637871+0000 mgr.smithi153.asomxr (mgr.14195) 1508 : audit 0 from='client.15310 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:16 smithi163 bash[22347]: cluster 2024-03-12T02:51:15.238201+0000 mgr.smithi153.asomxr (mgr.14195) 1509 : cluster 0 pgmap v1258: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:16 smithi163 bash[22347]: audit 2024-03-12T02:51:15.439703+0000 mon.smithi153 (mon.0) 1162 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:51:18.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:18 smithi153 bash[14340]: cluster 2024-03-12T02:51:17.238809+0000 mgr.smithi153.asomxr (mgr.14195) 1510 : cluster 0 pgmap v1259: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:18.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:18 smithi163 bash[22347]: cluster 2024-03-12T02:51:17.238809+0000 mgr.smithi153.asomxr (mgr.14195) 1510 : cluster 0 pgmap v1259: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:19.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:19 smithi153 bash[14340]: cluster 2024-03-12T02:51:19.239436+0000 mgr.smithi153.asomxr (mgr.14195) 1511 : cluster 0 pgmap v1260: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:19.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:19 smithi163 bash[22347]: cluster 2024-03-12T02:51:19.239436+0000 mgr.smithi153.asomxr (mgr.14195) 1511 : cluster 0 pgmap v1260: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:21.244 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:51:22.497 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:22 smithi153 bash[14340]: cluster 2024-03-12T02:51:21.240112+0000 mgr.smithi153.asomxr (mgr.14195) 1512 : cluster 0 pgmap v1261: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:22.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:22 smithi163 bash[22347]: cluster 2024-03-12T02:51:21.240112+0000 mgr.smithi153.asomxr (mgr.14195) 1512 : cluster 0 pgmap v1261: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:24.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:24 smithi153 bash[14340]: cluster 2024-03-12T02:51:23.240838+0000 mgr.smithi153.asomxr (mgr.14195) 1513 : cluster 0 pgmap v1262: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:24.530 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:51:24.531 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:51:24.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:24 smithi163 bash[22347]: cluster 2024-03-12T02:51:23.240838+0000 mgr.smithi153.asomxr (mgr.14195) 1513 : cluster 0 pgmap v1262: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:25.395 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:51:26.397 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:51:26.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:26 smithi153 bash[14340]: audit 2024-03-12T02:51:24.520011+0000 mgr.smithi153.asomxr (mgr.14195) 1514 : audit 0 from='client.15314 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:26.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:26 smithi153 bash[14340]: cluster 2024-03-12T02:51:25.241615+0000 mgr.smithi153.asomxr (mgr.14195) 1515 : cluster 0 pgmap v1263: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:26.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:26 smithi163 bash[22347]: audit 2024-03-12T02:51:24.520011+0000 mgr.smithi153.asomxr (mgr.14195) 1514 : audit 0 from='client.15314 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:26.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:26 smithi163 bash[22347]: cluster 2024-03-12T02:51:25.241615+0000 mgr.smithi153.asomxr (mgr.14195) 1515 : cluster 0 pgmap v1263: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:28.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:28 smithi153 bash[14340]: cluster 2024-03-12T02:51:27.242271+0000 mgr.smithi153.asomxr (mgr.14195) 1516 : cluster 0 pgmap v1264: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:28.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:28 smithi163 bash[22347]: cluster 2024-03-12T02:51:27.242271+0000 mgr.smithi153.asomxr (mgr.14195) 1516 : cluster 0 pgmap v1264: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:29.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:29 smithi163 bash[22347]: cluster 2024-03-12T02:51:29.243009+0000 mgr.smithi153.asomxr (mgr.14195) 1517 : cluster 0 pgmap v1265: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:29.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:29 smithi153 bash[14340]: cluster 2024-03-12T02:51:29.243009+0000 mgr.smithi153.asomxr (mgr.14195) 1517 : cluster 0 pgmap v1265: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:30.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:30 smithi153 bash[14340]: audit 2024-03-12T02:51:30.440616+0000 mon.smithi153 (mon.0) 1163 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:51:30.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:30 smithi163 bash[22347]: audit 2024-03-12T02:51:30.440616+0000 mon.smithi153 (mon.0) 1163 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:51:31.151 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:51:31.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:31 smithi153 bash[14340]: cluster 2024-03-12T02:51:31.243732+0000 mgr.smithi153.asomxr (mgr.14195) 1518 : cluster 0 pgmap v1266: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:31.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:31 smithi163 bash[22347]: cluster 2024-03-12T02:51:31.243732+0000 mgr.smithi153.asomxr (mgr.14195) 1518 : cluster 0 pgmap v1266: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:34.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:34 smithi153 bash[14340]: cluster 2024-03-12T02:51:33.244441+0000 mgr.smithi153.asomxr (mgr.14195) 1519 : cluster 0 pgmap v1267: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:34.680 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:51:34.681 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:51:34.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:34 smithi163 bash[22347]: cluster 2024-03-12T02:51:33.244441+0000 mgr.smithi153.asomxr (mgr.14195) 1519 : cluster 0 pgmap v1267: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:35.522 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:51:36.523 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:51:36.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:36 smithi153 bash[14340]: audit 2024-03-12T02:51:34.665798+0000 mgr.smithi153.asomxr (mgr.14195) 1520 : audit 0 from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:36.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:36 smithi153 bash[14340]: cluster 2024-03-12T02:51:35.245072+0000 mgr.smithi153.asomxr (mgr.14195) 1521 : cluster 0 pgmap v1268: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:36.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:36 smithi163 bash[22347]: audit 2024-03-12T02:51:34.665798+0000 mgr.smithi153.asomxr (mgr.14195) 1520 : audit 0 from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:36.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:36 smithi163 bash[22347]: cluster 2024-03-12T02:51:35.245072+0000 mgr.smithi153.asomxr (mgr.14195) 1521 : cluster 0 pgmap v1268: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:38.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:38 smithi153 bash[14340]: cluster 2024-03-12T02:51:37.245696+0000 mgr.smithi153.asomxr (mgr.14195) 1522 : cluster 0 pgmap v1269: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:38.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:38 smithi163 bash[22347]: cluster 2024-03-12T02:51:37.245696+0000 mgr.smithi153.asomxr (mgr.14195) 1522 : cluster 0 pgmap v1269: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:40.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:40 smithi153 bash[14340]: cluster 2024-03-12T02:51:39.246333+0000 mgr.smithi153.asomxr (mgr.14195) 1523 : cluster 0 pgmap v1270: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:40.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:40 smithi163 bash[22347]: cluster 2024-03-12T02:51:39.246333+0000 mgr.smithi153.asomxr (mgr.14195) 1523 : cluster 0 pgmap v1270: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:41.278 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:51:41.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:41 smithi153 bash[14340]: cluster 2024-03-12T02:51:41.246960+0000 mgr.smithi153.asomxr (mgr.14195) 1524 : cluster 0 pgmap v1271: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:41.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:41 smithi163 bash[22347]: cluster 2024-03-12T02:51:41.246960+0000 mgr.smithi153.asomxr (mgr.14195) 1524 : cluster 0 pgmap v1271: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:44.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:44 smithi153 bash[14340]: cluster 2024-03-12T02:51:43.247647+0000 mgr.smithi153.asomxr (mgr.14195) 1525 : cluster 0 pgmap v1272: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:44.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:44 smithi153 bash[14340]: audit 2024-03-12T02:51:43.328027+0000 mon.smithi153 (mon.0) 1164 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:51:44.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:44 smithi153 bash[14340]: audit 2024-03-12T02:51:43.706391+0000 mon.smithi153 (mon.0) 1165 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:51:44.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:44 smithi153 bash[14340]: audit 2024-03-12T02:51:43.716920+0000 mon.smithi153 (mon.0) 1166 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:51:44.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:44 smithi163 bash[22347]: cluster 2024-03-12T02:51:43.247647+0000 mgr.smithi153.asomxr (mgr.14195) 1525 : cluster 0 pgmap v1272: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:44.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:44 smithi163 bash[22347]: audit 2024-03-12T02:51:43.328027+0000 mon.smithi153 (mon.0) 1164 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:51:44.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:44 smithi163 bash[22347]: audit 2024-03-12T02:51:43.706391+0000 mon.smithi153 (mon.0) 1165 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:51:44.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:44 smithi163 bash[22347]: audit 2024-03-12T02:51:43.716920+0000 mon.smithi153 (mon.0) 1166 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:51:44.808 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:51:44.809 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:41:17.340660Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:41:17.339936Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:41:17.340163Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:41:17.340893Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:41:17.342296Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:41:17.342519Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:41:17.339700Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:41:17.339395Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:41:17.340388Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:41:17.341419Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:41:17.341116Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:51:45.667 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:51:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:46 smithi153 bash[14340]: audit 2024-03-12T02:51:44.793329+0000 mgr.smithi153.asomxr (mgr.14195) 1526 : audit 0 from='client.15322 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:46 smithi153 bash[14340]: cluster 2024-03-12T02:51:45.248279+0000 mgr.smithi153.asomxr (mgr.14195) 1527 : cluster 0 pgmap v1273: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:46 smithi153 bash[14340]: audit 2024-03-12T02:51:45.440558+0000 mon.smithi153 (mon.0) 1167 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:51:46.669 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:51:46.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:46 smithi163 bash[22347]: audit 2024-03-12T02:51:44.793329+0000 mgr.smithi153.asomxr (mgr.14195) 1526 : audit 0 from='client.15322 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:46.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:46 smithi163 bash[22347]: cluster 2024-03-12T02:51:45.248279+0000 mgr.smithi153.asomxr (mgr.14195) 1527 : cluster 0 pgmap v1273: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:46.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:46 smithi163 bash[22347]: audit 2024-03-12T02:51:45.440558+0000 mon.smithi153 (mon.0) 1167 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:51:48.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:48 smithi153 bash[14340]: cluster 2024-03-12T02:51:47.248900+0000 mgr.smithi153.asomxr (mgr.14195) 1528 : cluster 0 pgmap v1274: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:48.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:48 smithi163 bash[22347]: cluster 2024-03-12T02:51:47.248900+0000 mgr.smithi153.asomxr (mgr.14195) 1528 : cluster 0 pgmap v1274: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:49.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:49 smithi163 bash[22347]: cluster 2024-03-12T02:51:49.249543+0000 mgr.smithi153.asomxr (mgr.14195) 1529 : cluster 0 pgmap v1275: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:49.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:49 smithi153 bash[14340]: cluster 2024-03-12T02:51:49.249543+0000 mgr.smithi153.asomxr (mgr.14195) 1529 : cluster 0 pgmap v1275: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:51.242 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:51:52.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:51 smithi153 bash[14340]: audit 2024-03-12T02:51:50.732415+0000 mon.smithi153 (mon.0) 1168 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:51:52.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:51 smithi153 bash[14340]: audit 2024-03-12T02:51:50.740231+0000 mon.smithi153 (mon.0) 1169 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:51:52.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:51 smithi153 bash[14340]: cluster 2024-03-12T02:51:51.250046+0000 mgr.smithi153.asomxr (mgr.14195) 1530 : cluster 0 pgmap v1276: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:52.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:51 smithi153 bash[14340]: audit 2024-03-12T02:51:51.260469+0000 mon.smithi153 (mon.0) 1170 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:51:52.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:51 smithi153 bash[14340]: audit 2024-03-12T02:51:51.262057+0000 mon.smithi153 (mon.0) 1171 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:51:52.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:51 smithi153 bash[14340]: cluster 2024-03-12T02:51:51.264562+0000 mgr.smithi153.asomxr (mgr.14195) 1531 : cluster 0 pgmap v1277: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:52.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:51 smithi153 bash[14340]: cluster 2024-03-12T02:51:51.264791+0000 mgr.smithi153.asomxr (mgr.14195) 1532 : cluster 0 pgmap v1278: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:52.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:51 smithi153 bash[14340]: audit 2024-03-12T02:51:51.270361+0000 mon.smithi153 (mon.0) 1172 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:51:52.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:51 smithi153 bash[14340]: audit 2024-03-12T02:51:51.273704+0000 mon.smithi153 (mon.0) 1173 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:51:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:51 smithi163 bash[22347]: audit 2024-03-12T02:51:50.732415+0000 mon.smithi153 (mon.0) 1168 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:51:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:51 smithi163 bash[22347]: audit 2024-03-12T02:51:50.740231+0000 mon.smithi153 (mon.0) 1169 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:51:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:51 smithi163 bash[22347]: cluster 2024-03-12T02:51:51.250046+0000 mgr.smithi153.asomxr (mgr.14195) 1530 : cluster 0 pgmap v1276: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:51 smithi163 bash[22347]: audit 2024-03-12T02:51:51.260469+0000 mon.smithi153 (mon.0) 1170 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:51:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:51 smithi163 bash[22347]: audit 2024-03-12T02:51:51.262057+0000 mon.smithi153 (mon.0) 1171 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:51:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:51 smithi163 bash[22347]: cluster 2024-03-12T02:51:51.264562+0000 mgr.smithi153.asomxr (mgr.14195) 1531 : cluster 0 pgmap v1277: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:51 smithi163 bash[22347]: cluster 2024-03-12T02:51:51.264791+0000 mgr.smithi153.asomxr (mgr.14195) 1532 : cluster 0 pgmap v1278: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:51 smithi163 bash[22347]: audit 2024-03-12T02:51:51.270361+0000 mon.smithi153 (mon.0) 1172 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:51:52.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:51 smithi163 bash[22347]: audit 2024-03-12T02:51:51.273704+0000 mon.smithi153 (mon.0) 1173 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:51:54.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:54 smithi153 bash[14340]: cluster 2024-03-12T02:51:53.265396+0000 mgr.smithi153.asomxr (mgr.14195) 1533 : cluster 0 pgmap v1279: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:54.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:54 smithi163 bash[22347]: cluster 2024-03-12T02:51:53.265396+0000 mgr.smithi153.asomxr (mgr.14195) 1533 : cluster 0 pgmap v1279: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:54.788 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:51:54.788 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:51:55.606 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:51:56.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:56 smithi153 bash[14340]: audit 2024-03-12T02:51:54.773277+0000 mgr.smithi153.asomxr (mgr.14195) 1534 : audit 0 from='client.15326 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:56.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:56 smithi153 bash[14340]: cluster 2024-03-12T02:51:55.265967+0000 mgr.smithi153.asomxr (mgr.14195) 1535 : cluster 0 pgmap v1280: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:56.608 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:51:56.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:56 smithi163 bash[22347]: audit 2024-03-12T02:51:54.773277+0000 mgr.smithi153.asomxr (mgr.14195) 1534 : audit 0 from='client.15326 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:51:56.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:56 smithi163 bash[22347]: cluster 2024-03-12T02:51:55.265967+0000 mgr.smithi153.asomxr (mgr.14195) 1535 : cluster 0 pgmap v1280: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:58.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:51:58 smithi163 bash[22347]: cluster 2024-03-12T02:51:57.266557+0000 mgr.smithi153.asomxr (mgr.14195) 1536 : cluster 0 pgmap v1281: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:51:58.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:51:58 smithi153 bash[14340]: cluster 2024-03-12T02:51:57.266557+0000 mgr.smithi153.asomxr (mgr.14195) 1536 : cluster 0 pgmap v1281: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:00.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:00 smithi153 bash[14340]: cluster 2024-03-12T02:51:59.267171+0000 mgr.smithi153.asomxr (mgr.14195) 1537 : cluster 0 pgmap v1282: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:00.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:00 smithi163 bash[22347]: cluster 2024-03-12T02:51:59.267171+0000 mgr.smithi153.asomxr (mgr.14195) 1537 : cluster 0 pgmap v1282: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:01.354 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:52:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:01 smithi153 bash[14340]: audit 2024-03-12T02:52:00.441682+0000 mon.smithi153 (mon.0) 1174 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:52:01.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:01 smithi163 bash[22347]: audit 2024-03-12T02:52:00.441682+0000 mon.smithi153 (mon.0) 1174 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:52:02.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:02 smithi153 bash[14340]: cluster 2024-03-12T02:52:01.267770+0000 mgr.smithi153.asomxr (mgr.14195) 1538 : cluster 0 pgmap v1283: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:02.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:02 smithi163 bash[22347]: cluster 2024-03-12T02:52:01.267770+0000 mgr.smithi153.asomxr (mgr.14195) 1538 : cluster 0 pgmap v1283: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:04.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:04 smithi153 bash[14340]: cluster 2024-03-12T02:52:03.268396+0000 mgr.smithi153.asomxr (mgr.14195) 1539 : cluster 0 pgmap v1284: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:04.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:04 smithi163 bash[22347]: cluster 2024-03-12T02:52:03.268396+0000 mgr.smithi153.asomxr (mgr.14195) 1539 : cluster 0 pgmap v1284: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:04.831 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:52:04.831 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:52:05.682 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:52:06.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:06 smithi153 bash[14340]: audit 2024-03-12T02:52:04.813433+0000 mgr.smithi153.asomxr (mgr.14195) 1540 : audit 0 from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:06.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:06 smithi153 bash[14340]: cluster 2024-03-12T02:52:05.268793+0000 mgr.smithi153.asomxr (mgr.14195) 1541 : cluster 0 pgmap v1285: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:06.683 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:52:06.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:06 smithi163 bash[22347]: audit 2024-03-12T02:52:04.813433+0000 mgr.smithi153.asomxr (mgr.14195) 1540 : audit 0 from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:06.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:06 smithi163 bash[22347]: cluster 2024-03-12T02:52:05.268793+0000 mgr.smithi153.asomxr (mgr.14195) 1541 : cluster 0 pgmap v1285: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:08.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:08 smithi153 bash[14340]: cluster 2024-03-12T02:52:07.269377+0000 mgr.smithi153.asomxr (mgr.14195) 1542 : cluster 0 pgmap v1286: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:08.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:08 smithi163 bash[22347]: cluster 2024-03-12T02:52:07.269377+0000 mgr.smithi153.asomxr (mgr.14195) 1542 : cluster 0 pgmap v1286: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:10.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:10 smithi153 bash[14340]: cluster 2024-03-12T02:52:09.269985+0000 mgr.smithi153.asomxr (mgr.14195) 1543 : cluster 0 pgmap v1287: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:10.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:10 smithi163 bash[22347]: cluster 2024-03-12T02:52:09.269985+0000 mgr.smithi153.asomxr (mgr.14195) 1543 : cluster 0 pgmap v1287: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:11.437 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:52:12.568 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:12 smithi153 bash[14340]: cluster 2024-03-12T02:52:11.270536+0000 mgr.smithi153.asomxr (mgr.14195) 1544 : cluster 0 pgmap v1288: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:12.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:12 smithi163 bash[22347]: cluster 2024-03-12T02:52:11.270536+0000 mgr.smithi153.asomxr (mgr.14195) 1544 : cluster 0 pgmap v1288: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:14.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:14 smithi153 bash[14340]: cluster 2024-03-12T02:52:13.271084+0000 mgr.smithi153.asomxr (mgr.14195) 1545 : cluster 0 pgmap v1289: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:14.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:14 smithi163 bash[22347]: cluster 2024-03-12T02:52:13.271084+0000 mgr.smithi153.asomxr (mgr.14195) 1545 : cluster 0 pgmap v1289: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:14.804 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:52:14.804 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:52:15.640 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:15 smithi153 bash[14340]: audit 2024-03-12T02:52:14.788145+0000 mgr.smithi153.asomxr (mgr.14195) 1546 : audit 0 from='client.15334 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:15.640 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:15 smithi153 bash[14340]: cluster 2024-03-12T02:52:15.271608+0000 mgr.smithi153.asomxr (mgr.14195) 1547 : cluster 0 pgmap v1290: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:15.642 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:52:15.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:15 smithi163 bash[22347]: audit 2024-03-12T02:52:14.788145+0000 mgr.smithi153.asomxr (mgr.14195) 1546 : audit 0 from='client.15334 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:15.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:15 smithi163 bash[22347]: cluster 2024-03-12T02:52:15.271608+0000 mgr.smithi153.asomxr (mgr.14195) 1547 : cluster 0 pgmap v1290: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:16.642 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:52:16.655 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:16 smithi153 bash[14340]: audit 2024-03-12T02:52:15.441528+0000 mon.smithi153 (mon.0) 1175 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:52:16.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:16 smithi163 bash[22347]: audit 2024-03-12T02:52:15.441528+0000 mon.smithi153 (mon.0) 1175 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:52:17.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:17 smithi163 bash[22347]: cluster 2024-03-12T02:52:17.272316+0000 mgr.smithi153.asomxr (mgr.14195) 1548 : cluster 0 pgmap v1291: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:17.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:17 smithi153 bash[14340]: cluster 2024-03-12T02:52:17.272316+0000 mgr.smithi153.asomxr (mgr.14195) 1548 : cluster 0 pgmap v1291: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:20.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:20 smithi153 bash[14340]: cluster 2024-03-12T02:52:19.273111+0000 mgr.smithi153.asomxr (mgr.14195) 1549 : cluster 0 pgmap v1292: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:20.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:20 smithi163 bash[22347]: cluster 2024-03-12T02:52:19.273111+0000 mgr.smithi153.asomxr (mgr.14195) 1549 : cluster 0 pgmap v1292: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:21.388 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:52:22.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:22 smithi153 bash[14340]: cluster 2024-03-12T02:52:21.273814+0000 mgr.smithi153.asomxr (mgr.14195) 1550 : cluster 0 pgmap v1293: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:22.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:22 smithi163 bash[22347]: cluster 2024-03-12T02:52:21.273814+0000 mgr.smithi153.asomxr (mgr.14195) 1550 : cluster 0 pgmap v1293: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:24.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:24 smithi153 bash[14340]: cluster 2024-03-12T02:52:23.274405+0000 mgr.smithi153.asomxr (mgr.14195) 1551 : cluster 0 pgmap v1294: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:24.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:24 smithi163 bash[22347]: cluster 2024-03-12T02:52:23.274405+0000 mgr.smithi153.asomxr (mgr.14195) 1551 : cluster 0 pgmap v1294: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:24.758 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:52:24.758 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:52:25.578 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:52:25.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:25 smithi163 bash[22347]: audit 2024-03-12T02:52:24.744001+0000 mgr.smithi153.asomxr (mgr.14195) 1552 : audit 0 from='client.15338 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:25.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:25 smithi163 bash[22347]: cluster 2024-03-12T02:52:25.275008+0000 mgr.smithi153.asomxr (mgr.14195) 1553 : cluster 0 pgmap v1295: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:25.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:25 smithi153 bash[14340]: audit 2024-03-12T02:52:24.744001+0000 mgr.smithi153.asomxr (mgr.14195) 1552 : audit 0 from='client.15338 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:25.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:25 smithi153 bash[14340]: cluster 2024-03-12T02:52:25.275008+0000 mgr.smithi153.asomxr (mgr.14195) 1553 : cluster 0 pgmap v1295: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:26.580 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:52:27.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:27 smithi163 bash[22347]: cluster 2024-03-12T02:52:27.275595+0000 mgr.smithi153.asomxr (mgr.14195) 1554 : cluster 0 pgmap v1296: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:27.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:27 smithi153 bash[14340]: cluster 2024-03-12T02:52:27.275595+0000 mgr.smithi153.asomxr (mgr.14195) 1554 : cluster 0 pgmap v1296: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:30.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:30 smithi153 bash[14340]: cluster 2024-03-12T02:52:29.276188+0000 mgr.smithi153.asomxr (mgr.14195) 1555 : cluster 0 pgmap v1297: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:30.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:30 smithi163 bash[22347]: cluster 2024-03-12T02:52:29.276188+0000 mgr.smithi153.asomxr (mgr.14195) 1555 : cluster 0 pgmap v1297: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:31.330 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:52:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:31 smithi153 bash[14340]: audit 2024-03-12T02:52:30.442520+0000 mon.smithi153 (mon.0) 1176 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:52:31.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:31 smithi163 bash[22347]: audit 2024-03-12T02:52:30.442520+0000 mon.smithi153 (mon.0) 1176 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:52:32.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:32 smithi153 bash[14340]: cluster 2024-03-12T02:52:31.276789+0000 mgr.smithi153.asomxr (mgr.14195) 1556 : cluster 0 pgmap v1298: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:32.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:32 smithi163 bash[22347]: cluster 2024-03-12T02:52:31.276789+0000 mgr.smithi153.asomxr (mgr.14195) 1556 : cluster 0 pgmap v1298: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:34.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:34 smithi153 bash[14340]: cluster 2024-03-12T02:52:33.277240+0000 mgr.smithi153.asomxr (mgr.14195) 1557 : cluster 0 pgmap v1299: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:34.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:34 smithi163 bash[22347]: cluster 2024-03-12T02:52:33.277240+0000 mgr.smithi153.asomxr (mgr.14195) 1557 : cluster 0 pgmap v1299: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:34.745 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:52:34.745 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:52:35.640 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:52:36.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:36 smithi153 bash[14340]: audit 2024-03-12T02:52:34.728915+0000 mgr.smithi153.asomxr (mgr.14195) 1558 : audit 0 from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:36.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:36 smithi153 bash[14340]: cluster 2024-03-12T02:52:35.277793+0000 mgr.smithi153.asomxr (mgr.14195) 1559 : cluster 0 pgmap v1300: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:36.642 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:52:36.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:36 smithi163 bash[22347]: audit 2024-03-12T02:52:34.728915+0000 mgr.smithi153.asomxr (mgr.14195) 1558 : audit 0 from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:36.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:36 smithi163 bash[22347]: cluster 2024-03-12T02:52:35.277793+0000 mgr.smithi153.asomxr (mgr.14195) 1559 : cluster 0 pgmap v1300: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:38.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:38 smithi163 bash[22347]: cluster 2024-03-12T02:52:37.278459+0000 mgr.smithi153.asomxr (mgr.14195) 1560 : cluster 0 pgmap v1301: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:38.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:38 smithi153 bash[14340]: cluster 2024-03-12T02:52:37.278459+0000 mgr.smithi153.asomxr (mgr.14195) 1560 : cluster 0 pgmap v1301: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:39.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:39 smithi163 bash[22347]: cluster 2024-03-12T02:52:39.279122+0000 mgr.smithi153.asomxr (mgr.14195) 1561 : cluster 0 pgmap v1302: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:39.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:39 smithi153 bash[14340]: cluster 2024-03-12T02:52:39.279122+0000 mgr.smithi153.asomxr (mgr.14195) 1561 : cluster 0 pgmap v1302: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:41.385 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:52:41.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:41 smithi163 bash[22347]: cluster 2024-03-12T02:52:41.279847+0000 mgr.smithi153.asomxr (mgr.14195) 1562 : cluster 0 pgmap v1303: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:41.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:41 smithi153 bash[14340]: cluster 2024-03-12T02:52:41.279847+0000 mgr.smithi153.asomxr (mgr.14195) 1562 : cluster 0 pgmap v1303: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:44.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:44 smithi153 bash[14340]: cluster 2024-03-12T02:52:43.280461+0000 mgr.smithi153.asomxr (mgr.14195) 1563 : cluster 0 pgmap v1304: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:44.663 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:52:44.663 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:52:44.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:44 smithi163 bash[22347]: cluster 2024-03-12T02:52:43.280461+0000 mgr.smithi153.asomxr (mgr.14195) 1563 : cluster 0 pgmap v1304: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:45.495 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:52:46.497 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:52:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:46 smithi153 bash[14340]: audit 2024-03-12T02:52:44.648822+0000 mgr.smithi153.asomxr (mgr.14195) 1564 : audit 0 from='client.15346 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:46 smithi153 bash[14340]: cluster 2024-03-12T02:52:45.281106+0000 mgr.smithi153.asomxr (mgr.14195) 1565 : cluster 0 pgmap v1305: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:46.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:46 smithi153 bash[14340]: audit 2024-03-12T02:52:45.443064+0000 mon.smithi153 (mon.0) 1177 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:52:46.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:46 smithi163 bash[22347]: audit 2024-03-12T02:52:44.648822+0000 mgr.smithi153.asomxr (mgr.14195) 1564 : audit 0 from='client.15346 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:46.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:46 smithi163 bash[22347]: cluster 2024-03-12T02:52:45.281106+0000 mgr.smithi153.asomxr (mgr.14195) 1565 : cluster 0 pgmap v1305: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:46.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:46 smithi163 bash[22347]: audit 2024-03-12T02:52:45.443064+0000 mon.smithi153 (mon.0) 1177 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:52:48.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:48 smithi153 bash[14340]: cluster 2024-03-12T02:52:47.281783+0000 mgr.smithi153.asomxr (mgr.14195) 1566 : cluster 0 pgmap v1306: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:48.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:48 smithi163 bash[22347]: cluster 2024-03-12T02:52:47.281783+0000 mgr.smithi153.asomxr (mgr.14195) 1566 : cluster 0 pgmap v1306: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:50.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:50 smithi163 bash[22347]: cluster 2024-03-12T02:52:49.282496+0000 mgr.smithi153.asomxr (mgr.14195) 1567 : cluster 0 pgmap v1307: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:50.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:50 smithi153 bash[14340]: cluster 2024-03-12T02:52:49.282496+0000 mgr.smithi153.asomxr (mgr.14195) 1567 : cluster 0 pgmap v1307: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:51.259 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:52:51.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:51 smithi153 bash[14340]: cluster 2024-03-12T02:52:51.283083+0000 mgr.smithi153.asomxr (mgr.14195) 1568 : cluster 0 pgmap v1308: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:51.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:51 smithi153 bash[14340]: audit 2024-03-12T02:52:51.356361+0000 mon.smithi153 (mon.0) 1178 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:52:51.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:51 smithi163 bash[22347]: cluster 2024-03-12T02:52:51.283083+0000 mgr.smithi153.asomxr (mgr.14195) 1568 : cluster 0 pgmap v1308: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:51.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:51 smithi163 bash[22347]: audit 2024-03-12T02:52:51.356361+0000 mon.smithi153 (mon.0) 1178 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:52:52.633 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:52 smithi153 bash[14340]: audit 2024-03-12T02:52:51.880998+0000 mon.smithi153 (mon.0) 1179 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:52:52.633 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:52 smithi153 bash[14340]: audit 2024-03-12T02:52:51.882740+0000 mon.smithi153 (mon.0) 1180 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:52:52.633 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:52 smithi153 bash[14340]: cluster 2024-03-12T02:52:51.885409+0000 mgr.smithi153.asomxr (mgr.14195) 1569 : cluster 0 pgmap v1309: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:52.634 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:52 smithi153 bash[14340]: cluster 2024-03-12T02:52:51.885902+0000 mgr.smithi153.asomxr (mgr.14195) 1570 : cluster 0 pgmap v1310: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:52.634 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:52 smithi153 bash[14340]: audit 2024-03-12T02:52:51.891109+0000 mon.smithi153 (mon.0) 1181 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:52:52.634 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:52 smithi153 bash[14340]: audit 2024-03-12T02:52:51.895817+0000 mon.smithi153 (mon.0) 1182 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:52:52.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:52 smithi163 bash[22347]: audit 2024-03-12T02:52:51.880998+0000 mon.smithi153 (mon.0) 1179 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:52:52.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:52 smithi163 bash[22347]: audit 2024-03-12T02:52:51.882740+0000 mon.smithi153 (mon.0) 1180 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:52:52.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:52 smithi163 bash[22347]: cluster 2024-03-12T02:52:51.885409+0000 mgr.smithi153.asomxr (mgr.14195) 1569 : cluster 0 pgmap v1309: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:52.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:52 smithi163 bash[22347]: cluster 2024-03-12T02:52:51.885902+0000 mgr.smithi153.asomxr (mgr.14195) 1570 : cluster 0 pgmap v1310: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:52.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:52 smithi163 bash[22347]: audit 2024-03-12T02:52:51.891109+0000 mon.smithi153 (mon.0) 1181 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:52:52.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:52 smithi163 bash[22347]: audit 2024-03-12T02:52:51.895817+0000 mon.smithi153 (mon.0) 1182 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:52:54.692 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:52:54.692 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:52:55.530 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:52:55.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:55 smithi153 bash[14340]: cluster 2024-03-12T02:52:53.886507+0000 mgr.smithi153.asomxr (mgr.14195) 1571 : cluster 0 pgmap v1311: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:55.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:55 smithi163 bash[22347]: cluster 2024-03-12T02:52:53.886507+0000 mgr.smithi153.asomxr (mgr.14195) 1571 : cluster 0 pgmap v1311: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:56.531 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:52:56.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:56 smithi163 bash[22347]: audit 2024-03-12T02:52:54.680609+0000 mgr.smithi153.asomxr (mgr.14195) 1572 : audit 0 from='client.15350 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:56.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:56 smithi153 bash[14340]: audit 2024-03-12T02:52:54.680609+0000 mgr.smithi153.asomxr (mgr.14195) 1572 : audit 0 from='client.15350 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:52:57.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:57 smithi163 bash[22347]: cluster 2024-03-12T02:52:55.887290+0000 mgr.smithi153.asomxr (mgr.14195) 1573 : cluster 0 pgmap v1312: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:57.825 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:57 smithi153 bash[14340]: cluster 2024-03-12T02:52:55.887290+0000 mgr.smithi153.asomxr (mgr.14195) 1573 : cluster 0 pgmap v1312: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:52:59 smithi153 bash[14340]: cluster 2024-03-12T02:52:57.887959+0000 mgr.smithi153.asomxr (mgr.14195) 1574 : cluster 0 pgmap v1313: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:52:59.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:52:59 smithi163 bash[22347]: cluster 2024-03-12T02:52:57.887959+0000 mgr.smithi153.asomxr (mgr.14195) 1574 : cluster 0 pgmap v1313: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:01.280 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:53:01.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:01 smithi153 bash[14340]: cluster 2024-03-12T02:52:59.888670+0000 mgr.smithi153.asomxr (mgr.14195) 1575 : cluster 0 pgmap v1314: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:01.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:01 smithi153 bash[14340]: audit 2024-03-12T02:53:00.443639+0000 mon.smithi153 (mon.0) 1183 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:53:01.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:01 smithi163 bash[22347]: cluster 2024-03-12T02:52:59.888670+0000 mgr.smithi153.asomxr (mgr.14195) 1575 : cluster 0 pgmap v1314: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:01.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:01 smithi163 bash[22347]: audit 2024-03-12T02:53:00.443639+0000 mon.smithi153 (mon.0) 1183 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:53:03.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:03 smithi153 bash[14340]: cluster 2024-03-12T02:53:01.889469+0000 mgr.smithi153.asomxr (mgr.14195) 1576 : cluster 0 pgmap v1315: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:03.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:03 smithi163 bash[22347]: cluster 2024-03-12T02:53:01.889469+0000 mgr.smithi153.asomxr (mgr.14195) 1576 : cluster 0 pgmap v1315: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:04.706 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:53:04.706 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:53:05.434 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:53:05.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:05 smithi163 bash[22347]: cluster 2024-03-12T02:53:03.890144+0000 mgr.smithi153.asomxr (mgr.14195) 1577 : cluster 0 pgmap v1316: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:05.914 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:05 smithi153 bash[14340]: cluster 2024-03-12T02:53:03.890144+0000 mgr.smithi153.asomxr (mgr.14195) 1577 : cluster 0 pgmap v1316: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:06.435 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:53:06.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:06 smithi163 bash[22347]: audit 2024-03-12T02:53:04.691816+0000 mgr.smithi153.asomxr (mgr.14195) 1578 : audit 0 from='client.15354 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:06.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:06 smithi153 bash[14340]: audit 2024-03-12T02:53:04.691816+0000 mgr.smithi153.asomxr (mgr.14195) 1578 : audit 0 from='client.15354 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:07.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:07 smithi163 bash[22347]: cluster 2024-03-12T02:53:05.890792+0000 mgr.smithi153.asomxr (mgr.14195) 1579 : cluster 0 pgmap v1317: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:07.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:07 smithi153 bash[14340]: cluster 2024-03-12T02:53:05.890792+0000 mgr.smithi153.asomxr (mgr.14195) 1579 : cluster 0 pgmap v1317: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:09.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:09 smithi163 bash[22347]: cluster 2024-03-12T02:53:07.891415+0000 mgr.smithi153.asomxr (mgr.14195) 1580 : cluster 0 pgmap v1318: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:09.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:09 smithi153 bash[14340]: cluster 2024-03-12T02:53:07.891415+0000 mgr.smithi153.asomxr (mgr.14195) 1580 : cluster 0 pgmap v1318: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:11.192 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:53:11.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:11 smithi153 bash[14340]: cluster 2024-03-12T02:53:09.892036+0000 mgr.smithi153.asomxr (mgr.14195) 1581 : cluster 0 pgmap v1319: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:11.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:11 smithi163 bash[22347]: cluster 2024-03-12T02:53:09.892036+0000 mgr.smithi153.asomxr (mgr.14195) 1581 : cluster 0 pgmap v1319: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:12.672 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:12 smithi153 bash[14340]: cluster 2024-03-12T02:53:11.892713+0000 mgr.smithi153.asomxr (mgr.14195) 1582 : cluster 0 pgmap v1320: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:12.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:12 smithi163 bash[22347]: cluster 2024-03-12T02:53:11.892713+0000 mgr.smithi153.asomxr (mgr.14195) 1582 : cluster 0 pgmap v1320: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:14.696 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:53:14.697 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:53:15.480 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:15 smithi153 bash[14340]: cluster 2024-03-12T02:53:13.893364+0000 mgr.smithi153.asomxr (mgr.14195) 1583 : cluster 0 pgmap v1321: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:15.481 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:53:15.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:15 smithi163 bash[22347]: cluster 2024-03-12T02:53:13.893364+0000 mgr.smithi153.asomxr (mgr.14195) 1583 : cluster 0 pgmap v1321: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:16.482 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:53:16.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:16 smithi163 bash[22347]: audit 2024-03-12T02:53:14.680920+0000 mgr.smithi153.asomxr (mgr.14195) 1584 : audit 0 from='client.15358 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:16 smithi163 bash[22347]: audit 2024-03-12T02:53:15.443686+0000 mon.smithi153 (mon.0) 1184 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:53:16.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:16 smithi153 bash[14340]: audit 2024-03-12T02:53:14.680920+0000 mgr.smithi153.asomxr (mgr.14195) 1584 : audit 0 from='client.15358 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:16.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:16 smithi153 bash[14340]: audit 2024-03-12T02:53:15.443686+0000 mon.smithi153 (mon.0) 1184 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:53:17.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:17 smithi163 bash[22347]: cluster 2024-03-12T02:53:15.894015+0000 mgr.smithi153.asomxr (mgr.14195) 1585 : cluster 0 pgmap v1322: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:17.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:17 smithi153 bash[14340]: cluster 2024-03-12T02:53:15.894015+0000 mgr.smithi153.asomxr (mgr.14195) 1585 : cluster 0 pgmap v1322: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:19.697 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:19 smithi163 bash[22347]: cluster 2024-03-12T02:53:17.894618+0000 mgr.smithi153.asomxr (mgr.14195) 1586 : cluster 0 pgmap v1323: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:19.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:19 smithi153 bash[14340]: cluster 2024-03-12T02:53:17.894618+0000 mgr.smithi153.asomxr (mgr.14195) 1586 : cluster 0 pgmap v1323: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:20.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:20 smithi153 bash[14340]: cluster 2024-03-12T02:53:19.895227+0000 mgr.smithi153.asomxr (mgr.14195) 1587 : cluster 0 pgmap v1324: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:20.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:20 smithi163 bash[22347]: cluster 2024-03-12T02:53:19.895227+0000 mgr.smithi153.asomxr (mgr.14195) 1587 : cluster 0 pgmap v1324: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:21.234 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:53:23.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:22 smithi163 bash[22347]: cluster 2024-03-12T02:53:21.895835+0000 mgr.smithi153.asomxr (mgr.14195) 1588 : cluster 0 pgmap v1325: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:23.269 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:22 smithi153 bash[14340]: cluster 2024-03-12T02:53:21.895835+0000 mgr.smithi153.asomxr (mgr.14195) 1588 : cluster 0 pgmap v1325: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:24.628 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:53:24.628 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:53:25.436 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:53:25.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:25 smithi163 bash[22347]: cluster 2024-03-12T02:53:23.896449+0000 mgr.smithi153.asomxr (mgr.14195) 1589 : cluster 0 pgmap v1326: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:25.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:25 smithi153 bash[14340]: cluster 2024-03-12T02:53:23.896449+0000 mgr.smithi153.asomxr (mgr.14195) 1589 : cluster 0 pgmap v1326: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:26.437 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:53:26.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:26 smithi163 bash[22347]: audit 2024-03-12T02:53:24.609317+0000 mgr.smithi153.asomxr (mgr.14195) 1590 : audit 0 from='client.15362 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:26.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:26 smithi153 bash[14340]: audit 2024-03-12T02:53:24.609317+0000 mgr.smithi153.asomxr (mgr.14195) 1590 : audit 0 from='client.15362 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:27.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:27 smithi163 bash[22347]: cluster 2024-03-12T02:53:25.897057+0000 mgr.smithi153.asomxr (mgr.14195) 1591 : cluster 0 pgmap v1327: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:27.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:27 smithi153 bash[14340]: cluster 2024-03-12T02:53:25.897057+0000 mgr.smithi153.asomxr (mgr.14195) 1591 : cluster 0 pgmap v1327: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:29.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:29 smithi163 bash[22347]: cluster 2024-03-12T02:53:27.897652+0000 mgr.smithi153.asomxr (mgr.14195) 1592 : cluster 0 pgmap v1328: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:29.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:29 smithi153 bash[14340]: cluster 2024-03-12T02:53:27.897652+0000 mgr.smithi153.asomxr (mgr.14195) 1592 : cluster 0 pgmap v1328: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:31.189 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:53:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:31 smithi153 bash[14340]: cluster 2024-03-12T02:53:29.898202+0000 mgr.smithi153.asomxr (mgr.14195) 1593 : cluster 0 pgmap v1329: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:31.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:31 smithi153 bash[14340]: audit 2024-03-12T02:53:30.444362+0000 mon.smithi153 (mon.0) 1185 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:53:31.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:31 smithi163 bash[22347]: cluster 2024-03-12T02:53:29.898202+0000 mgr.smithi153.asomxr (mgr.14195) 1593 : cluster 0 pgmap v1329: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:31.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:31 smithi163 bash[22347]: audit 2024-03-12T02:53:30.444362+0000 mon.smithi153 (mon.0) 1185 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:53:32.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:32 smithi153 bash[14340]: cluster 2024-03-12T02:53:31.898832+0000 mgr.smithi153.asomxr (mgr.14195) 1594 : cluster 0 pgmap v1330: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:32.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:32 smithi163 bash[22347]: cluster 2024-03-12T02:53:31.898832+0000 mgr.smithi153.asomxr (mgr.14195) 1594 : cluster 0 pgmap v1330: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:34.597 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:53:34.597 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:53:35.376 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:53:35.627 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:35 smithi153 bash[14340]: cluster 2024-03-12T02:53:33.899475+0000 mgr.smithi153.asomxr (mgr.14195) 1595 : cluster 0 pgmap v1331: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:35.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:35 smithi163 bash[22347]: cluster 2024-03-12T02:53:33.899475+0000 mgr.smithi153.asomxr (mgr.14195) 1595 : cluster 0 pgmap v1331: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:36.377 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:53:36.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:36 smithi163 bash[22347]: audit 2024-03-12T02:53:34.580315+0000 mgr.smithi153.asomxr (mgr.14195) 1596 : audit 0 from='client.15366 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:36.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:36 smithi153 bash[14340]: audit 2024-03-12T02:53:34.580315+0000 mgr.smithi153.asomxr (mgr.14195) 1596 : audit 0 from='client.15366 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:37.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:37 smithi163 bash[22347]: cluster 2024-03-12T02:53:35.900077+0000 mgr.smithi153.asomxr (mgr.14195) 1597 : cluster 0 pgmap v1332: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:37.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:37 smithi153 bash[14340]: cluster 2024-03-12T02:53:35.900077+0000 mgr.smithi153.asomxr (mgr.14195) 1597 : cluster 0 pgmap v1332: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:39.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:39 smithi163 bash[22347]: cluster 2024-03-12T02:53:37.900660+0000 mgr.smithi153.asomxr (mgr.14195) 1598 : cluster 0 pgmap v1333: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:39.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:39 smithi153 bash[14340]: cluster 2024-03-12T02:53:37.900660+0000 mgr.smithi153.asomxr (mgr.14195) 1598 : cluster 0 pgmap v1333: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:41.125 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:53:41.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:41 smithi153 bash[14340]: cluster 2024-03-12T02:53:39.901291+0000 mgr.smithi153.asomxr (mgr.14195) 1599 : cluster 0 pgmap v1334: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:41.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:41 smithi163 bash[22347]: cluster 2024-03-12T02:53:39.901291+0000 mgr.smithi153.asomxr (mgr.14195) 1599 : cluster 0 pgmap v1334: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:42.549 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:42 smithi153 bash[14340]: cluster 2024-03-12T02:53:41.901797+0000 mgr.smithi153.asomxr (mgr.14195) 1600 : cluster 0 pgmap v1335: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:42.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:42 smithi163 bash[22347]: cluster 2024-03-12T02:53:41.901797+0000 mgr.smithi153.asomxr (mgr.14195) 1600 : cluster 0 pgmap v1335: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:44.684 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:53:44.684 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:53:45.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:45 smithi163 bash[22347]: cluster 2024-03-12T02:53:43.902376+0000 mgr.smithi153.asomxr (mgr.14195) 1601 : cluster 0 pgmap v1336: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:45.540 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:53:45.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:45 smithi153 bash[14340]: cluster 2024-03-12T02:53:43.902376+0000 mgr.smithi153.asomxr (mgr.14195) 1601 : cluster 0 pgmap v1336: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:46.541 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:53:46.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:46 smithi163 bash[22347]: audit 2024-03-12T02:53:44.668124+0000 mgr.smithi153.asomxr (mgr.14195) 1602 : audit 0 from='client.15370 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:46.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:46 smithi163 bash[22347]: audit 2024-03-12T02:53:45.444643+0000 mon.smithi153 (mon.0) 1186 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:53:46.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:46 smithi153 bash[14340]: audit 2024-03-12T02:53:44.668124+0000 mgr.smithi153.asomxr (mgr.14195) 1602 : audit 0 from='client.15370 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:46.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:46 smithi153 bash[14340]: audit 2024-03-12T02:53:45.444643+0000 mon.smithi153 (mon.0) 1186 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:53:47.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:47 smithi163 bash[22347]: cluster 2024-03-12T02:53:45.902822+0000 mgr.smithi153.asomxr (mgr.14195) 1603 : cluster 0 pgmap v1337: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:47.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:47 smithi153 bash[14340]: cluster 2024-03-12T02:53:45.902822+0000 mgr.smithi153.asomxr (mgr.14195) 1603 : cluster 0 pgmap v1337: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:49.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:49 smithi163 bash[22347]: cluster 2024-03-12T02:53:47.903381+0000 mgr.smithi153.asomxr (mgr.14195) 1604 : cluster 0 pgmap v1338: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:49.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:49 smithi153 bash[14340]: cluster 2024-03-12T02:53:47.903381+0000 mgr.smithi153.asomxr (mgr.14195) 1604 : cluster 0 pgmap v1338: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:51.297 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:53:51.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:51 smithi163 bash[22347]: cluster 2024-03-12T02:53:49.904113+0000 mgr.smithi153.asomxr (mgr.14195) 1605 : cluster 0 pgmap v1339: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:51.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:51 smithi153 bash[14340]: cluster 2024-03-12T02:53:49.904113+0000 mgr.smithi153.asomxr (mgr.14195) 1605 : cluster 0 pgmap v1339: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:52.697 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:52 smithi153 bash[14340]: cluster 2024-03-12T02:53:51.904582+0000 mgr.smithi153.asomxr (mgr.14195) 1606 : cluster 0 pgmap v1340: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:52.698 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:52 smithi153 bash[14340]: audit 2024-03-12T02:53:51.988506+0000 mon.smithi153 (mon.0) 1187 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:53:52.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:52 smithi163 bash[22347]: cluster 2024-03-12T02:53:51.904582+0000 mgr.smithi153.asomxr (mgr.14195) 1606 : cluster 0 pgmap v1340: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:52.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:52 smithi163 bash[22347]: audit 2024-03-12T02:53:51.988506+0000 mon.smithi153 (mon.0) 1187 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:53:53.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:53 smithi163 bash[22347]: audit 2024-03-12T02:53:52.364151+0000 mon.smithi153 (mon.0) 1188 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:53:53.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:53 smithi163 bash[22347]: audit 2024-03-12T02:53:52.370764+0000 mon.smithi153 (mon.0) 1189 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:53:53.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:53 smithi153 bash[14340]: audit 2024-03-12T02:53:52.364151+0000 mon.smithi153 (mon.0) 1188 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:53:53.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:53 smithi153 bash[14340]: audit 2024-03-12T02:53:52.370764+0000 mon.smithi153 (mon.0) 1189 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:53:54.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:54 smithi163 bash[22347]: cluster 2024-03-12T02:53:53.905313+0000 mgr.smithi153.asomxr (mgr.14195) 1607 : cluster 0 pgmap v1341: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:54.814 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:53:54.814 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:43:25.240728Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:43:25.240956Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:43:25.242694Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:43:25.242504Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:43:25.242885Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:43:25.241396Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:43:25.241591Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:43:25.241155Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:43:25.241783Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:53:54.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:54 smithi153 bash[14340]: cluster 2024-03-12T02:53:53.905313+0000 mgr.smithi153.asomxr (mgr.14195) 1607 : cluster 0 pgmap v1341: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:55.619 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:53:55.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:55 smithi163 bash[22347]: audit 2024-03-12T02:53:54.799125+0000 mgr.smithi153.asomxr (mgr.14195) 1608 : audit 0 from='client.15374 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:55.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:55 smithi153 bash[14340]: audit 2024-03-12T02:53:54.799125+0000 mgr.smithi153.asomxr (mgr.14195) 1608 : audit 0 from='client.15374 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:53:56.620 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:53:56.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:56 smithi163 bash[22347]: cluster 2024-03-12T02:53:55.906031+0000 mgr.smithi153.asomxr (mgr.14195) 1609 : cluster 0 pgmap v1342: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:56.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:56 smithi153 bash[14340]: cluster 2024-03-12T02:53:55.906031+0000 mgr.smithi153.asomxr (mgr.14195) 1609 : cluster 0 pgmap v1342: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:59 smithi153 bash[14340]: cluster 2024-03-12T02:53:57.906606+0000 mgr.smithi153.asomxr (mgr.14195) 1610 : cluster 0 pgmap v1343: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:59 smithi153 bash[14340]: audit 2024-03-12T02:53:58.268974+0000 mon.smithi153 (mon.0) 1190 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:53:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:59 smithi153 bash[14340]: audit 2024-03-12T02:53:58.277308+0000 mon.smithi153 (mon.0) 1191 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:53:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:59 smithi153 bash[14340]: audit 2024-03-12T02:53:58.799414+0000 mon.smithi153 (mon.0) 1192 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:53:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:59 smithi153 bash[14340]: audit 2024-03-12T02:53:58.801355+0000 mon.smithi153 (mon.0) 1193 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:53:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:59 smithi153 bash[14340]: audit 2024-03-12T02:53:58.811672+0000 mon.smithi153 (mon.0) 1194 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:53:59.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:53:59 smithi153 bash[14340]: audit 2024-03-12T02:53:58.819091+0000 mon.smithi153 (mon.0) 1195 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:53:59.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:59 smithi163 bash[22347]: cluster 2024-03-12T02:53:57.906606+0000 mgr.smithi153.asomxr (mgr.14195) 1610 : cluster 0 pgmap v1343: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:53:59.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:59 smithi163 bash[22347]: audit 2024-03-12T02:53:58.268974+0000 mon.smithi153 (mon.0) 1190 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:53:59.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:59 smithi163 bash[22347]: audit 2024-03-12T02:53:58.277308+0000 mon.smithi153 (mon.0) 1191 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:53:59.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:59 smithi163 bash[22347]: audit 2024-03-12T02:53:58.799414+0000 mon.smithi153 (mon.0) 1192 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:53:59.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:59 smithi163 bash[22347]: audit 2024-03-12T02:53:58.801355+0000 mon.smithi153 (mon.0) 1193 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:53:59.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:59 smithi163 bash[22347]: audit 2024-03-12T02:53:58.811672+0000 mon.smithi153 (mon.0) 1194 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:53:59.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:53:59 smithi163 bash[22347]: audit 2024-03-12T02:53:58.819091+0000 mon.smithi153 (mon.0) 1195 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:54:00.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:00 smithi163 bash[22347]: cluster 2024-03-12T02:53:58.804632+0000 mgr.smithi153.asomxr (mgr.14195) 1611 : cluster 0 pgmap v1344: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:00.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:00 smithi163 bash[22347]: cluster 2024-03-12T02:53:58.805081+0000 mgr.smithi153.asomxr (mgr.14195) 1612 : cluster 0 pgmap v1345: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:00.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:00 smithi153 bash[14340]: cluster 2024-03-12T02:53:58.804632+0000 mgr.smithi153.asomxr (mgr.14195) 1611 : cluster 0 pgmap v1344: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:00.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:00 smithi153 bash[14340]: cluster 2024-03-12T02:53:58.805081+0000 mgr.smithi153.asomxr (mgr.14195) 1612 : cluster 0 pgmap v1345: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:01.375 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:54:01.409 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:01 smithi153 bash[14340]: audit 2024-03-12T02:54:00.445053+0000 mon.smithi153 (mon.0) 1196 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:54:01.409 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:01 smithi153 bash[14340]: cluster 2024-03-12T02:54:00.805631+0000 mgr.smithi153.asomxr (mgr.14195) 1613 : cluster 0 pgmap v1346: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:01.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:01 smithi163 bash[22347]: audit 2024-03-12T02:54:00.445053+0000 mon.smithi153 (mon.0) 1196 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:54:01.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:01 smithi163 bash[22347]: cluster 2024-03-12T02:54:00.805631+0000 mgr.smithi153.asomxr (mgr.14195) 1613 : cluster 0 pgmap v1346: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:04.563 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:54:04.564 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:54:04.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:04 smithi153 bash[14340]: cluster 2024-03-12T02:54:02.806207+0000 mgr.smithi153.asomxr (mgr.14195) 1614 : cluster 0 pgmap v1347: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:04.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:04 smithi163 bash[22347]: cluster 2024-03-12T02:54:02.806207+0000 mgr.smithi153.asomxr (mgr.14195) 1614 : cluster 0 pgmap v1347: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:05.434 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:54:06.436 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:54:06.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:06 smithi153 bash[14340]: audit 2024-03-12T02:54:04.546182+0000 mgr.smithi153.asomxr (mgr.14195) 1615 : audit 0 from='client.15378 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:06.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:06 smithi153 bash[14340]: cluster 2024-03-12T02:54:04.806844+0000 mgr.smithi153.asomxr (mgr.14195) 1616 : cluster 0 pgmap v1348: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:06.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:06 smithi163 bash[22347]: audit 2024-03-12T02:54:04.546182+0000 mgr.smithi153.asomxr (mgr.14195) 1615 : audit 0 from='client.15378 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:06.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:06 smithi163 bash[22347]: cluster 2024-03-12T02:54:04.806844+0000 mgr.smithi153.asomxr (mgr.14195) 1616 : cluster 0 pgmap v1348: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:07.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:07 smithi163 bash[22347]: cluster 2024-03-12T02:54:06.807482+0000 mgr.smithi153.asomxr (mgr.14195) 1617 : cluster 0 pgmap v1349: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:07.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:07 smithi153 bash[14340]: cluster 2024-03-12T02:54:06.807482+0000 mgr.smithi153.asomxr (mgr.14195) 1617 : cluster 0 pgmap v1349: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:10.729 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:10 smithi163 bash[22347]: cluster 2024-03-12T02:54:08.808143+0000 mgr.smithi153.asomxr (mgr.14195) 1618 : cluster 0 pgmap v1350: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:10.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:10 smithi153 bash[14340]: cluster 2024-03-12T02:54:08.808143+0000 mgr.smithi153.asomxr (mgr.14195) 1618 : cluster 0 pgmap v1350: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:11.195 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:54:11.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:11 smithi153 bash[14340]: cluster 2024-03-12T02:54:10.808808+0000 mgr.smithi153.asomxr (mgr.14195) 1619 : cluster 0 pgmap v1351: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:11.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:11 smithi163 bash[22347]: cluster 2024-03-12T02:54:10.808808+0000 mgr.smithi153.asomxr (mgr.14195) 1619 : cluster 0 pgmap v1351: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:14.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:14 smithi153 bash[14340]: cluster 2024-03-12T02:54:12.809413+0000 mgr.smithi153.asomxr (mgr.14195) 1620 : cluster 0 pgmap v1352: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:14.622 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:54:14.622 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:54:14.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:14 smithi163 bash[22347]: cluster 2024-03-12T02:54:12.809413+0000 mgr.smithi153.asomxr (mgr.14195) 1620 : cluster 0 pgmap v1352: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:15.375 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:54:15.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:15 smithi163 bash[22347]: audit 2024-03-12T02:54:14.606439+0000 mgr.smithi153.asomxr (mgr.14195) 1621 : audit 0 from='client.15382 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:15.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:15 smithi163 bash[22347]: cluster 2024-03-12T02:54:14.809973+0000 mgr.smithi153.asomxr (mgr.14195) 1622 : cluster 0 pgmap v1353: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:15.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:15 smithi153 bash[14340]: audit 2024-03-12T02:54:14.606439+0000 mgr.smithi153.asomxr (mgr.14195) 1621 : audit 0 from='client.15382 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:15.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:15 smithi153 bash[14340]: cluster 2024-03-12T02:54:14.809973+0000 mgr.smithi153.asomxr (mgr.14195) 1622 : cluster 0 pgmap v1353: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:16.376 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:54:16.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:16 smithi163 bash[22347]: audit 2024-03-12T02:54:15.445402+0000 mon.smithi153 (mon.0) 1197 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:54:16.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:16 smithi153 bash[14340]: audit 2024-03-12T02:54:15.445402+0000 mon.smithi153 (mon.0) 1197 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:54:17.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:17 smithi163 bash[22347]: cluster 2024-03-12T02:54:16.810638+0000 mgr.smithi153.asomxr (mgr.14195) 1623 : cluster 0 pgmap v1354: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:17.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:17 smithi153 bash[14340]: cluster 2024-03-12T02:54:16.810638+0000 mgr.smithi153.asomxr (mgr.14195) 1623 : cluster 0 pgmap v1354: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:19.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:19 smithi163 bash[22347]: cluster 2024-03-12T02:54:18.811258+0000 mgr.smithi153.asomxr (mgr.14195) 1624 : cluster 0 pgmap v1355: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:19.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:19 smithi153 bash[14340]: cluster 2024-03-12T02:54:18.811258+0000 mgr.smithi153.asomxr (mgr.14195) 1624 : cluster 0 pgmap v1355: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:21.131 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:54:22.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:21 smithi163 bash[22347]: cluster 2024-03-12T02:54:20.811913+0000 mgr.smithi153.asomxr (mgr.14195) 1625 : cluster 0 pgmap v1356: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:22.244 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:21 smithi153 bash[14340]: cluster 2024-03-12T02:54:20.811913+0000 mgr.smithi153.asomxr (mgr.14195) 1625 : cluster 0 pgmap v1356: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:24.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:24 smithi153 bash[14340]: cluster 2024-03-12T02:54:22.812556+0000 mgr.smithi153.asomxr (mgr.14195) 1626 : cluster 0 pgmap v1357: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:24.607 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:54:24.608 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:54:24.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:24 smithi163 bash[22347]: cluster 2024-03-12T02:54:22.812556+0000 mgr.smithi153.asomxr (mgr.14195) 1626 : cluster 0 pgmap v1357: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:25.401 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:54:25.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:25 smithi163 bash[22347]: audit 2024-03-12T02:54:24.591083+0000 mgr.smithi153.asomxr (mgr.14195) 1627 : audit 0 from='client.15386 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:25.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:25 smithi163 bash[22347]: cluster 2024-03-12T02:54:24.813207+0000 mgr.smithi153.asomxr (mgr.14195) 1628 : cluster 0 pgmap v1358: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:25.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:25 smithi153 bash[14340]: audit 2024-03-12T02:54:24.591083+0000 mgr.smithi153.asomxr (mgr.14195) 1627 : audit 0 from='client.15386 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:25.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:25 smithi153 bash[14340]: cluster 2024-03-12T02:54:24.813207+0000 mgr.smithi153.asomxr (mgr.14195) 1628 : cluster 0 pgmap v1358: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:26.402 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:54:28.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:27 smithi163 bash[22347]: cluster 2024-03-12T02:54:26.813797+0000 mgr.smithi153.asomxr (mgr.14195) 1629 : cluster 0 pgmap v1359: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:28.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:27 smithi153 bash[14340]: cluster 2024-03-12T02:54:26.813797+0000 mgr.smithi153.asomxr (mgr.14195) 1629 : cluster 0 pgmap v1359: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:29.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:29 smithi163 bash[22347]: cluster 2024-03-12T02:54:28.814444+0000 mgr.smithi153.asomxr (mgr.14195) 1630 : cluster 0 pgmap v1360: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:29.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:29 smithi153 bash[14340]: cluster 2024-03-12T02:54:28.814444+0000 mgr.smithi153.asomxr (mgr.14195) 1630 : cluster 0 pgmap v1360: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:30.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:30 smithi153 bash[14340]: audit 2024-03-12T02:54:30.445938+0000 mon.smithi153 (mon.0) 1198 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:54:30.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:30 smithi163 bash[22347]: audit 2024-03-12T02:54:30.445938+0000 mon.smithi153 (mon.0) 1198 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:54:31.159 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:54:31.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:31 smithi153 bash[14340]: cluster 2024-03-12T02:54:30.815099+0000 mgr.smithi153.asomxr (mgr.14195) 1631 : cluster 0 pgmap v1361: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:31.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:31 smithi163 bash[22347]: cluster 2024-03-12T02:54:30.815099+0000 mgr.smithi153.asomxr (mgr.14195) 1631 : cluster 0 pgmap v1361: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:34.219 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:33 smithi153 bash[14340]: cluster 2024-03-12T02:54:32.815727+0000 mgr.smithi153.asomxr (mgr.14195) 1632 : cluster 0 pgmap v1362: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:34.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:33 smithi163 bash[22347]: cluster 2024-03-12T02:54:32.815727+0000 mgr.smithi153.asomxr (mgr.14195) 1632 : cluster 0 pgmap v1362: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:34.673 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:54:34.673 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:54:35.500 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:54:36.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:35 smithi163 bash[22347]: audit 2024-03-12T02:54:34.658116+0000 mgr.smithi153.asomxr (mgr.14195) 1633 : audit 0 from='client.15390 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:36.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:35 smithi163 bash[22347]: cluster 2024-03-12T02:54:34.816369+0000 mgr.smithi153.asomxr (mgr.14195) 1634 : cluster 0 pgmap v1363: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:36.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:35 smithi153 bash[14340]: audit 2024-03-12T02:54:34.658116+0000 mgr.smithi153.asomxr (mgr.14195) 1633 : audit 0 from='client.15390 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:36.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:35 smithi153 bash[14340]: cluster 2024-03-12T02:54:34.816369+0000 mgr.smithi153.asomxr (mgr.14195) 1634 : cluster 0 pgmap v1363: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:36.502 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:54:38.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:37 smithi163 bash[22347]: cluster 2024-03-12T02:54:36.816890+0000 mgr.smithi153.asomxr (mgr.14195) 1635 : cluster 0 pgmap v1364: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:38.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:37 smithi153 bash[14340]: cluster 2024-03-12T02:54:36.816890+0000 mgr.smithi153.asomxr (mgr.14195) 1635 : cluster 0 pgmap v1364: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:39.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:39 smithi163 bash[22347]: cluster 2024-03-12T02:54:38.817553+0000 mgr.smithi153.asomxr (mgr.14195) 1636 : cluster 0 pgmap v1365: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:39.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:39 smithi153 bash[14340]: cluster 2024-03-12T02:54:38.817553+0000 mgr.smithi153.asomxr (mgr.14195) 1636 : cluster 0 pgmap v1365: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:41.261 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:54:42.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:41 smithi163 bash[22347]: cluster 2024-03-12T02:54:40.818148+0000 mgr.smithi153.asomxr (mgr.14195) 1637 : cluster 0 pgmap v1366: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:42.265 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:41 smithi153 bash[14340]: cluster 2024-03-12T02:54:40.818148+0000 mgr.smithi153.asomxr (mgr.14195) 1637 : cluster 0 pgmap v1366: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:44.208 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:43 smithi153 bash[14340]: cluster 2024-03-12T02:54:42.818757+0000 mgr.smithi153.asomxr (mgr.14195) 1638 : cluster 0 pgmap v1367: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:44.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:43 smithi163 bash[22347]: cluster 2024-03-12T02:54:42.818757+0000 mgr.smithi153.asomxr (mgr.14195) 1638 : cluster 0 pgmap v1367: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:44.663 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:54:44.663 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:54:45.500 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:54:46.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:45 smithi163 bash[22347]: audit 2024-03-12T02:54:44.645897+0000 mgr.smithi153.asomxr (mgr.14195) 1639 : audit 0 from='client.15394 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:46.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:45 smithi163 bash[22347]: cluster 2024-03-12T02:54:44.819477+0000 mgr.smithi153.asomxr (mgr.14195) 1640 : cluster 0 pgmap v1368: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:46.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:45 smithi163 bash[22347]: audit 2024-03-12T02:54:45.446420+0000 mon.smithi153 (mon.0) 1199 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:54:46.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:45 smithi153 bash[14340]: audit 2024-03-12T02:54:44.645897+0000 mgr.smithi153.asomxr (mgr.14195) 1639 : audit 0 from='client.15394 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:45 smithi153 bash[14340]: cluster 2024-03-12T02:54:44.819477+0000 mgr.smithi153.asomxr (mgr.14195) 1640 : cluster 0 pgmap v1368: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:46.324 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:45 smithi153 bash[14340]: audit 2024-03-12T02:54:45.446420+0000 mon.smithi153 (mon.0) 1199 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:54:46.501 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:54:48.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:47 smithi163 bash[22347]: cluster 2024-03-12T02:54:46.820090+0000 mgr.smithi153.asomxr (mgr.14195) 1641 : cluster 0 pgmap v1369: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:48.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:47 smithi153 bash[14340]: cluster 2024-03-12T02:54:46.820090+0000 mgr.smithi153.asomxr (mgr.14195) 1641 : cluster 0 pgmap v1369: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:49.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:49 smithi163 bash[22347]: cluster 2024-03-12T02:54:48.820734+0000 mgr.smithi153.asomxr (mgr.14195) 1642 : cluster 0 pgmap v1370: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:49.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:49 smithi153 bash[14340]: cluster 2024-03-12T02:54:48.820734+0000 mgr.smithi153.asomxr (mgr.14195) 1642 : cluster 0 pgmap v1370: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:51.250 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:54:52.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:51 smithi163 bash[22347]: cluster 2024-03-12T02:54:50.821542+0000 mgr.smithi153.asomxr (mgr.14195) 1643 : cluster 0 pgmap v1371: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:52.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:51 smithi153 bash[14340]: cluster 2024-03-12T02:54:50.821542+0000 mgr.smithi153.asomxr (mgr.14195) 1643 : cluster 0 pgmap v1371: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:54.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:53 smithi163 bash[22347]: cluster 2024-03-12T02:54:52.822322+0000 mgr.smithi153.asomxr (mgr.14195) 1644 : cluster 0 pgmap v1372: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:54.292 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:53 smithi153 bash[14340]: cluster 2024-03-12T02:54:52.822322+0000 mgr.smithi153.asomxr (mgr.14195) 1644 : cluster 0 pgmap v1372: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:54.745 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:54:54.745 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:54:55.559 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:54:56.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:55 smithi163 bash[22347]: audit 2024-03-12T02:54:54.730646+0000 mgr.smithi153.asomxr (mgr.14195) 1645 : audit 0 from='client.15398 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:56.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:55 smithi163 bash[22347]: cluster 2024-03-12T02:54:54.822980+0000 mgr.smithi153.asomxr (mgr.14195) 1646 : cluster 0 pgmap v1373: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:56.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:55 smithi153 bash[14340]: audit 2024-03-12T02:54:54.730646+0000 mgr.smithi153.asomxr (mgr.14195) 1645 : audit 0 from='client.15398 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:54:56.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:55 smithi153 bash[14340]: cluster 2024-03-12T02:54:54.822980+0000 mgr.smithi153.asomxr (mgr.14195) 1646 : cluster 0 pgmap v1373: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:56.559 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:54:58.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:57 smithi163 bash[22347]: cluster 2024-03-12T02:54:56.823637+0000 mgr.smithi153.asomxr (mgr.14195) 1647 : cluster 0 pgmap v1374: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:57 smithi153 bash[14340]: cluster 2024-03-12T02:54:56.823637+0000 mgr.smithi153.asomxr (mgr.14195) 1647 : cluster 0 pgmap v1374: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:59.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:59 smithi163 bash[22347]: cluster 2024-03-12T02:54:58.824270+0000 mgr.smithi153.asomxr (mgr.14195) 1648 : cluster 0 pgmap v1375: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:59.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:54:59 smithi163 bash[22347]: audit 2024-03-12T02:54:58.909115+0000 mon.smithi153 (mon.0) 1200 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:54:59.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:59 smithi153 bash[14340]: cluster 2024-03-12T02:54:58.824270+0000 mgr.smithi153.asomxr (mgr.14195) 1648 : cluster 0 pgmap v1375: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:54:59.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:54:59 smithi153 bash[14340]: audit 2024-03-12T02:54:58.909115+0000 mon.smithi153 (mon.0) 1200 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:55:00.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:00 smithi163 bash[22347]: audit 2024-03-12T02:54:59.489551+0000 mon.smithi153 (mon.0) 1201 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:55:00.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:00 smithi163 bash[22347]: audit 2024-03-12T02:54:59.491297+0000 mon.smithi153 (mon.0) 1202 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:55:00.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:00 smithi163 bash[22347]: cluster 2024-03-12T02:54:59.494290+0000 mgr.smithi153.asomxr (mgr.14195) 1649 : cluster 0 pgmap v1376: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:00.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:00 smithi163 bash[22347]: cluster 2024-03-12T02:54:59.494644+0000 mgr.smithi153.asomxr (mgr.14195) 1650 : cluster 0 pgmap v1377: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:00.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:00 smithi163 bash[22347]: audit 2024-03-12T02:54:59.501451+0000 mon.smithi153 (mon.0) 1203 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:55:00.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:00 smithi163 bash[22347]: audit 2024-03-12T02:54:59.505366+0000 mon.smithi153 (mon.0) 1204 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:55:00.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:00 smithi153 bash[14340]: audit 2024-03-12T02:54:59.489551+0000 mon.smithi153 (mon.0) 1201 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:55:00.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:00 smithi153 bash[14340]: audit 2024-03-12T02:54:59.491297+0000 mon.smithi153 (mon.0) 1202 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:55:00.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:00 smithi153 bash[14340]: cluster 2024-03-12T02:54:59.494290+0000 mgr.smithi153.asomxr (mgr.14195) 1649 : cluster 0 pgmap v1376: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:00.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:00 smithi153 bash[14340]: cluster 2024-03-12T02:54:59.494644+0000 mgr.smithi153.asomxr (mgr.14195) 1650 : cluster 0 pgmap v1377: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:00.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:00 smithi153 bash[14340]: audit 2024-03-12T02:54:59.501451+0000 mon.smithi153 (mon.0) 1203 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:55:00.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:00 smithi153 bash[14340]: audit 2024-03-12T02:54:59.505366+0000 mon.smithi153 (mon.0) 1204 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:55:01.316 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:55:01.731 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:01 smithi163 bash[22347]: audit 2024-03-12T02:55:00.446818+0000 mon.smithi153 (mon.0) 1205 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:55:01.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:01 smithi153 bash[14340]: audit 2024-03-12T02:55:00.446818+0000 mon.smithi153 (mon.0) 1205 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:55:02.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:02 smithi153 bash[14340]: cluster 2024-03-12T02:55:01.495250+0000 mgr.smithi153.asomxr (mgr.14195) 1651 : cluster 0 pgmap v1378: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:02.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:02 smithi163 bash[22347]: cluster 2024-03-12T02:55:01.495250+0000 mgr.smithi153.asomxr (mgr.14195) 1651 : cluster 0 pgmap v1378: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:04.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:04 smithi163 bash[22347]: cluster 2024-03-12T02:55:03.495885+0000 mgr.smithi153.asomxr (mgr.14195) 1652 : cluster 0 pgmap v1379: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:04.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:04 smithi153 bash[14340]: cluster 2024-03-12T02:55:03.495885+0000 mgr.smithi153.asomxr (mgr.14195) 1652 : cluster 0 pgmap v1379: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:04.851 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:55:04.852 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:55:05.630 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:05 smithi153 bash[14340]: audit 2024-03-12T02:55:04.834942+0000 mgr.smithi153.asomxr (mgr.14195) 1653 : audit 0 from='client.15402 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:05.632 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:55:05.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:05 smithi163 bash[22347]: audit 2024-03-12T02:55:04.834942+0000 mgr.smithi153.asomxr (mgr.14195) 1653 : audit 0 from='client.15402 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:06.633 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:55:06.644 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:06 smithi153 bash[14340]: cluster 2024-03-12T02:55:05.496491+0000 mgr.smithi153.asomxr (mgr.14195) 1654 : cluster 0 pgmap v1380: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:06.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:06 smithi163 bash[22347]: cluster 2024-03-12T02:55:05.496491+0000 mgr.smithi153.asomxr (mgr.14195) 1654 : cluster 0 pgmap v1380: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:08.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:08 smithi153 bash[14340]: cluster 2024-03-12T02:55:07.497130+0000 mgr.smithi153.asomxr (mgr.14195) 1655 : cluster 0 pgmap v1381: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:08.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:08 smithi163 bash[22347]: cluster 2024-03-12T02:55:07.497130+0000 mgr.smithi153.asomxr (mgr.14195) 1655 : cluster 0 pgmap v1381: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:10.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:10 smithi153 bash[14340]: cluster 2024-03-12T02:55:09.497752+0000 mgr.smithi153.asomxr (mgr.14195) 1656 : cluster 0 pgmap v1382: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:10.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:10 smithi163 bash[22347]: cluster 2024-03-12T02:55:09.497752+0000 mgr.smithi153.asomxr (mgr.14195) 1656 : cluster 0 pgmap v1382: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:11.378 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:55:12.718 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:12 smithi153 bash[14340]: cluster 2024-03-12T02:55:11.498225+0000 mgr.smithi153.asomxr (mgr.14195) 1657 : cluster 0 pgmap v1383: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:12.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:12 smithi163 bash[22347]: cluster 2024-03-12T02:55:11.498225+0000 mgr.smithi153.asomxr (mgr.14195) 1657 : cluster 0 pgmap v1383: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:14.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:14 smithi153 bash[14340]: cluster 2024-03-12T02:55:13.498961+0000 mgr.smithi153.asomxr (mgr.14195) 1658 : cluster 0 pgmap v1384: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:14.696 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:55:14.696 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:55:14.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:14 smithi163 bash[22347]: cluster 2024-03-12T02:55:13.498961+0000 mgr.smithi153.asomxr (mgr.14195) 1658 : cluster 0 pgmap v1384: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:15.468 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:55:15.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:15 smithi163 bash[22347]: audit 2024-03-12T02:55:14.678874+0000 mgr.smithi153.asomxr (mgr.14195) 1659 : audit 0 from='client.15406 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:15.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:15 smithi153 bash[14340]: audit 2024-03-12T02:55:14.678874+0000 mgr.smithi153.asomxr (mgr.14195) 1659 : audit 0 from='client.15406 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:16.469 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:55:16.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:16 smithi163 bash[22347]: audit 2024-03-12T02:55:15.446886+0000 mon.smithi153 (mon.0) 1206 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:55:16.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:16 smithi163 bash[22347]: cluster 2024-03-12T02:55:15.499596+0000 mgr.smithi153.asomxr (mgr.14195) 1660 : cluster 0 pgmap v1385: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:16.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:16 smithi153 bash[14340]: audit 2024-03-12T02:55:15.446886+0000 mon.smithi153 (mon.0) 1206 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:55:16.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:16 smithi153 bash[14340]: cluster 2024-03-12T02:55:15.499596+0000 mgr.smithi153.asomxr (mgr.14195) 1660 : cluster 0 pgmap v1385: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:18.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:18 smithi153 bash[14340]: cluster 2024-03-12T02:55:17.500200+0000 mgr.smithi153.asomxr (mgr.14195) 1661 : cluster 0 pgmap v1386: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:18.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:18 smithi163 bash[22347]: cluster 2024-03-12T02:55:17.500200+0000 mgr.smithi153.asomxr (mgr.14195) 1661 : cluster 0 pgmap v1386: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:20.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:20 smithi153 bash[14340]: cluster 2024-03-12T02:55:19.500821+0000 mgr.smithi153.asomxr (mgr.14195) 1662 : cluster 0 pgmap v1387: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:20.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:20 smithi163 bash[22347]: cluster 2024-03-12T02:55:19.500821+0000 mgr.smithi153.asomxr (mgr.14195) 1662 : cluster 0 pgmap v1387: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:21.217 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:55:22.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:22 smithi153 bash[14340]: cluster 2024-03-12T02:55:21.501346+0000 mgr.smithi153.asomxr (mgr.14195) 1663 : cluster 0 pgmap v1388: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:22.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:22 smithi163 bash[22347]: cluster 2024-03-12T02:55:21.501346+0000 mgr.smithi153.asomxr (mgr.14195) 1663 : cluster 0 pgmap v1388: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:24.532 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:55:24.532 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:55:24.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:24 smithi163 bash[22347]: cluster 2024-03-12T02:55:23.501887+0000 mgr.smithi153.asomxr (mgr.14195) 1664 : cluster 0 pgmap v1389: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:24.791 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:24 smithi153 bash[14340]: cluster 2024-03-12T02:55:23.501887+0000 mgr.smithi153.asomxr (mgr.14195) 1664 : cluster 0 pgmap v1389: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:25.322 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:55:25.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:25 smithi153 bash[14340]: audit 2024-03-12T02:55:24.517559+0000 mgr.smithi153.asomxr (mgr.14195) 1665 : audit 0 from='client.15410 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:25.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:25 smithi163 bash[22347]: audit 2024-03-12T02:55:24.517559+0000 mgr.smithi153.asomxr (mgr.14195) 1665 : audit 0 from='client.15410 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:26.323 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:55:26.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:26 smithi163 bash[22347]: cluster 2024-03-12T02:55:25.502523+0000 mgr.smithi153.asomxr (mgr.14195) 1666 : cluster 0 pgmap v1390: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:26.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:26 smithi153 bash[14340]: cluster 2024-03-12T02:55:25.502523+0000 mgr.smithi153.asomxr (mgr.14195) 1666 : cluster 0 pgmap v1390: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:28.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:28 smithi153 bash[14340]: cluster 2024-03-12T02:55:27.503035+0000 mgr.smithi153.asomxr (mgr.14195) 1667 : cluster 0 pgmap v1391: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:28.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:28 smithi163 bash[22347]: cluster 2024-03-12T02:55:27.503035+0000 mgr.smithi153.asomxr (mgr.14195) 1667 : cluster 0 pgmap v1391: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:30.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:30 smithi153 bash[14340]: cluster 2024-03-12T02:55:29.503634+0000 mgr.smithi153.asomxr (mgr.14195) 1668 : cluster 0 pgmap v1392: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:30.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:30 smithi153 bash[14340]: audit 2024-03-12T02:55:30.447587+0000 mon.smithi153 (mon.0) 1207 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:55:30.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:30 smithi163 bash[22347]: cluster 2024-03-12T02:55:29.503634+0000 mgr.smithi153.asomxr (mgr.14195) 1668 : cluster 0 pgmap v1392: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:30.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:30 smithi163 bash[22347]: audit 2024-03-12T02:55:30.447587+0000 mon.smithi153 (mon.0) 1207 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:55:31.076 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:55:32.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:32 smithi163 bash[22347]: cluster 2024-03-12T02:55:31.504186+0000 mgr.smithi153.asomxr (mgr.14195) 1669 : cluster 0 pgmap v1393: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:32.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:32 smithi153 bash[14340]: cluster 2024-03-12T02:55:31.504186+0000 mgr.smithi153.asomxr (mgr.14195) 1669 : cluster 0 pgmap v1393: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:34.526 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:55:34.526 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:55:34.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:34 smithi163 bash[22347]: cluster 2024-03-12T02:55:33.504588+0000 mgr.smithi153.asomxr (mgr.14195) 1670 : cluster 0 pgmap v1394: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:34.747 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:34 smithi153 bash[14340]: cluster 2024-03-12T02:55:33.504588+0000 mgr.smithi153.asomxr (mgr.14195) 1670 : cluster 0 pgmap v1394: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:35.301 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:55:35.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:35 smithi153 bash[14340]: audit 2024-03-12T02:55:34.513483+0000 mgr.smithi153.asomxr (mgr.14195) 1671 : audit 0 from='client.15414 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:35.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:35 smithi163 bash[22347]: audit 2024-03-12T02:55:34.513483+0000 mgr.smithi153.asomxr (mgr.14195) 1671 : audit 0 from='client.15414 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:36.302 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:55:36.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:36 smithi153 bash[14340]: cluster 2024-03-12T02:55:35.505165+0000 mgr.smithi153.asomxr (mgr.14195) 1672 : cluster 0 pgmap v1395: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:36.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:36 smithi163 bash[22347]: cluster 2024-03-12T02:55:35.505165+0000 mgr.smithi153.asomxr (mgr.14195) 1672 : cluster 0 pgmap v1395: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:38.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:38 smithi153 bash[14340]: cluster 2024-03-12T02:55:37.505774+0000 mgr.smithi153.asomxr (mgr.14195) 1673 : cluster 0 pgmap v1396: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:38.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:38 smithi163 bash[22347]: cluster 2024-03-12T02:55:37.505774+0000 mgr.smithi153.asomxr (mgr.14195) 1673 : cluster 0 pgmap v1396: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:40.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:40 smithi153 bash[14340]: cluster 2024-03-12T02:55:39.506483+0000 mgr.smithi153.asomxr (mgr.14195) 1674 : cluster 0 pgmap v1397: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:40.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:40 smithi163 bash[22347]: cluster 2024-03-12T02:55:39.506483+0000 mgr.smithi153.asomxr (mgr.14195) 1674 : cluster 0 pgmap v1397: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:41.054 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:55:42.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:42 smithi153 bash[14340]: cluster 2024-03-12T02:55:41.507121+0000 mgr.smithi153.asomxr (mgr.14195) 1675 : cluster 0 pgmap v1398: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:42.987 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:42 smithi163 bash[22347]: cluster 2024-03-12T02:55:41.507121+0000 mgr.smithi153.asomxr (mgr.14195) 1675 : cluster 0 pgmap v1398: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:44.395 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:55:44.395 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:55:44.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:44 smithi163 bash[22347]: cluster 2024-03-12T02:55:43.507814+0000 mgr.smithi153.asomxr (mgr.14195) 1676 : cluster 0 pgmap v1399: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:44.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:44 smithi153 bash[14340]: cluster 2024-03-12T02:55:43.507814+0000 mgr.smithi153.asomxr (mgr.14195) 1676 : cluster 0 pgmap v1399: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:45.201 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:55:45.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:45 smithi153 bash[14340]: audit 2024-03-12T02:55:44.380721+0000 mgr.smithi153.asomxr (mgr.14195) 1677 : audit 0 from='client.15418 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:45.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:45 smithi163 bash[22347]: audit 2024-03-12T02:55:44.380721+0000 mgr.smithi153.asomxr (mgr.14195) 1677 : audit 0 from='client.15418 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:46.201 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:55:46.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:46 smithi153 bash[14340]: audit 2024-03-12T02:55:45.448106+0000 mon.smithi153 (mon.0) 1208 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:55:46.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:46 smithi153 bash[14340]: cluster 2024-03-12T02:55:45.508548+0000 mgr.smithi153.asomxr (mgr.14195) 1678 : cluster 0 pgmap v1400: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:46.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:46 smithi163 bash[22347]: audit 2024-03-12T02:55:45.448106+0000 mon.smithi153 (mon.0) 1208 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:55:46.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:46 smithi163 bash[22347]: cluster 2024-03-12T02:55:45.508548+0000 mgr.smithi153.asomxr (mgr.14195) 1678 : cluster 0 pgmap v1400: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:48.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:48 smithi153 bash[14340]: cluster 2024-03-12T02:55:47.509169+0000 mgr.smithi153.asomxr (mgr.14195) 1679 : cluster 0 pgmap v1401: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:48.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:48 smithi163 bash[22347]: cluster 2024-03-12T02:55:47.509169+0000 mgr.smithi153.asomxr (mgr.14195) 1679 : cluster 0 pgmap v1401: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:50.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:50 smithi153 bash[14340]: cluster 2024-03-12T02:55:49.509865+0000 mgr.smithi153.asomxr (mgr.14195) 1680 : cluster 0 pgmap v1402: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:50.949 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:55:50.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:50 smithi163 bash[22347]: cluster 2024-03-12T02:55:49.509865+0000 mgr.smithi153.asomxr (mgr.14195) 1680 : cluster 0 pgmap v1402: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:52.910 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:52 smithi153 bash[14340]: cluster 2024-03-12T02:55:51.510505+0000 mgr.smithi153.asomxr (mgr.14195) 1681 : cluster 0 pgmap v1403: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:52.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:52 smithi163 bash[22347]: cluster 2024-03-12T02:55:51.510505+0000 mgr.smithi153.asomxr (mgr.14195) 1681 : cluster 0 pgmap v1403: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:54.355 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:55:54.356 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:55:54.636 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:54 smithi153 bash[14340]: cluster 2024-03-12T02:55:53.510931+0000 mgr.smithi153.asomxr (mgr.14195) 1682 : cluster 0 pgmap v1404: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:54.636 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:54 smithi153 bash[14340]: audit 2024-03-12T02:55:54.340314+0000 mgr.smithi153.asomxr (mgr.14195) 1683 : audit 0 from='client.15422 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:54.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:54 smithi163 bash[22347]: cluster 2024-03-12T02:55:53.510931+0000 mgr.smithi153.asomxr (mgr.14195) 1682 : cluster 0 pgmap v1404: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:54.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:54 smithi163 bash[22347]: audit 2024-03-12T02:55:54.340314+0000 mgr.smithi153.asomxr (mgr.14195) 1683 : audit 0 from='client.15422 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:55:55.234 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:55:56.235 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:55:56.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:56 smithi153 bash[14340]: cluster 2024-03-12T02:55:55.511578+0000 mgr.smithi153.asomxr (mgr.14195) 1684 : cluster 0 pgmap v1405: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:56.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:56 smithi163 bash[22347]: cluster 2024-03-12T02:55:55.511578+0000 mgr.smithi153.asomxr (mgr.14195) 1684 : cluster 0 pgmap v1405: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:58.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:55:58 smithi153 bash[14340]: cluster 2024-03-12T02:55:57.512175+0000 mgr.smithi153.asomxr (mgr.14195) 1685 : cluster 0 pgmap v1406: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:55:58.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:55:58 smithi163 bash[22347]: cluster 2024-03-12T02:55:57.512175+0000 mgr.smithi153.asomxr (mgr.14195) 1685 : cluster 0 pgmap v1406: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:00.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:00 smithi163 bash[22347]: cluster 2024-03-12T02:55:59.512837+0000 mgr.smithi153.asomxr (mgr.14195) 1686 : cluster 0 pgmap v1407: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:00.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:00 smithi163 bash[22347]: audit 2024-03-12T02:55:59.601564+0000 mon.smithi153 (mon.0) 1209 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:56:00.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:00 smithi163 bash[22347]: audit 2024-03-12T02:56:00.178088+0000 mon.smithi153 (mon.0) 1210 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:56:00.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:00 smithi163 bash[22347]: audit 2024-03-12T02:56:00.179734+0000 mon.smithi153 (mon.0) 1211 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:56:00.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:00 smithi163 bash[22347]: cluster 2024-03-12T02:56:00.182552+0000 mgr.smithi153.asomxr (mgr.14195) 1687 : cluster 0 pgmap v1408: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:00.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:00 smithi163 bash[22347]: audit 2024-03-12T02:56:00.188077+0000 mon.smithi153 (mon.0) 1212 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:56:00.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:00 smithi163 bash[22347]: audit 2024-03-12T02:56:00.192646+0000 mon.smithi153 (mon.0) 1213 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:56:00.986 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:00 smithi163 bash[22347]: audit 2024-03-12T02:56:00.448752+0000 mon.smithi153 (mon.0) 1214 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:56:00.987 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:56:01.018 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:00 smithi153 bash[14340]: cluster 2024-03-12T02:55:59.512837+0000 mgr.smithi153.asomxr (mgr.14195) 1686 : cluster 0 pgmap v1407: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:01.018 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:00 smithi153 bash[14340]: audit 2024-03-12T02:55:59.601564+0000 mon.smithi153 (mon.0) 1209 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:56:01.018 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:00 smithi153 bash[14340]: audit 2024-03-12T02:56:00.178088+0000 mon.smithi153 (mon.0) 1210 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:56:01.018 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:00 smithi153 bash[14340]: audit 2024-03-12T02:56:00.179734+0000 mon.smithi153 (mon.0) 1211 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:56:01.018 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:00 smithi153 bash[14340]: cluster 2024-03-12T02:56:00.182552+0000 mgr.smithi153.asomxr (mgr.14195) 1687 : cluster 0 pgmap v1408: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:01.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:00 smithi153 bash[14340]: audit 2024-03-12T02:56:00.188077+0000 mon.smithi153 (mon.0) 1212 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:56:01.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:00 smithi153 bash[14340]: audit 2024-03-12T02:56:00.192646+0000 mon.smithi153 (mon.0) 1213 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:56:01.019 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:00 smithi153 bash[14340]: audit 2024-03-12T02:56:00.448752+0000 mon.smithi153 (mon.0) 1214 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:56:03.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:03 smithi153 bash[14340]: cluster 2024-03-12T02:56:02.183143+0000 mgr.smithi153.asomxr (mgr.14195) 1688 : cluster 0 pgmap v1409: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:03.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:03 smithi163 bash[22347]: cluster 2024-03-12T02:56:02.183143+0000 mgr.smithi153.asomxr (mgr.14195) 1688 : cluster 0 pgmap v1409: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:04.095 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:56:04.095 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:56:04.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:04 smithi153 bash[14340]: audit 2024-03-12T02:56:04.080167+0000 mgr.smithi153.asomxr (mgr.14195) 1689 : audit 0 from='client.15426 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:04.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:04 smithi153 bash[14340]: cluster 2024-03-12T02:56:04.183778+0000 mgr.smithi153.asomxr (mgr.14195) 1690 : cluster 0 pgmap v1410: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:04.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:04 smithi163 bash[22347]: audit 2024-03-12T02:56:04.080167+0000 mgr.smithi153.asomxr (mgr.14195) 1689 : audit 0 from='client.15426 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:04.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:04 smithi163 bash[22347]: cluster 2024-03-12T02:56:04.183778+0000 mgr.smithi153.asomxr (mgr.14195) 1690 : cluster 0 pgmap v1410: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:04.889 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:56:05.890 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:56:07.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:07 smithi153 bash[14340]: cluster 2024-03-12T02:56:06.184425+0000 mgr.smithi153.asomxr (mgr.14195) 1691 : cluster 0 pgmap v1411: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:07.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:07 smithi163 bash[22347]: cluster 2024-03-12T02:56:06.184425+0000 mgr.smithi153.asomxr (mgr.14195) 1691 : cluster 0 pgmap v1411: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:09.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:09 smithi153 bash[14340]: cluster 2024-03-12T02:56:08.185044+0000 mgr.smithi153.asomxr (mgr.14195) 1692 : cluster 0 pgmap v1412: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:09.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:09 smithi163 bash[22347]: cluster 2024-03-12T02:56:08.185044+0000 mgr.smithi153.asomxr (mgr.14195) 1692 : cluster 0 pgmap v1412: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:10.642 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:56:11.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:11 smithi153 bash[14340]: cluster 2024-03-12T02:56:10.185731+0000 mgr.smithi153.asomxr (mgr.14195) 1693 : cluster 0 pgmap v1413: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:11.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:11 smithi163 bash[22347]: cluster 2024-03-12T02:56:10.185731+0000 mgr.smithi153.asomxr (mgr.14195) 1693 : cluster 0 pgmap v1413: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:13.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:13 smithi163 bash[22347]: cluster 2024-03-12T02:56:12.186381+0000 mgr.smithi153.asomxr (mgr.14195) 1694 : cluster 0 pgmap v1414: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:13.569 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:13 smithi153 bash[14340]: cluster 2024-03-12T02:56:12.186381+0000 mgr.smithi153.asomxr (mgr.14195) 1694 : cluster 0 pgmap v1414: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:14.009 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:56:14.009 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:56:14.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:14 smithi153 bash[14340]: audit 2024-03-12T02:56:13.995888+0000 mgr.smithi153.asomxr (mgr.14195) 1695 : audit 0 from='client.15430 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:14.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:14 smithi153 bash[14340]: cluster 2024-03-12T02:56:14.186780+0000 mgr.smithi153.asomxr (mgr.14195) 1696 : cluster 0 pgmap v1415: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:14.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:14 smithi163 bash[22347]: audit 2024-03-12T02:56:13.995888+0000 mgr.smithi153.asomxr (mgr.14195) 1695 : audit 0 from='client.15430 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:14.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:14 smithi163 bash[22347]: cluster 2024-03-12T02:56:14.186780+0000 mgr.smithi153.asomxr (mgr.14195) 1696 : cluster 0 pgmap v1415: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:14.865 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:56:15.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:15 smithi153 bash[14340]: audit 2024-03-12T02:56:15.449086+0000 mon.smithi153 (mon.0) 1215 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:56:15.866 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:56:15.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:15 smithi163 bash[22347]: audit 2024-03-12T02:56:15.449086+0000 mon.smithi153 (mon.0) 1215 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:56:16.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:16 smithi153 bash[14340]: cluster 2024-03-12T02:56:16.187386+0000 mgr.smithi153.asomxr (mgr.14195) 1697 : cluster 0 pgmap v1416: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:16.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:16 smithi163 bash[22347]: cluster 2024-03-12T02:56:16.187386+0000 mgr.smithi153.asomxr (mgr.14195) 1697 : cluster 0 pgmap v1416: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:19.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:19 smithi153 bash[14340]: cluster 2024-03-12T02:56:18.188025+0000 mgr.smithi153.asomxr (mgr.14195) 1698 : cluster 0 pgmap v1417: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:19.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:19 smithi163 bash[22347]: cluster 2024-03-12T02:56:18.188025+0000 mgr.smithi153.asomxr (mgr.14195) 1698 : cluster 0 pgmap v1417: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:20.617 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:56:21.513 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:21 smithi153 bash[14340]: cluster 2024-03-12T02:56:20.188722+0000 mgr.smithi153.asomxr (mgr.14195) 1699 : cluster 0 pgmap v1418: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:21.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:21 smithi163 bash[22347]: cluster 2024-03-12T02:56:20.188722+0000 mgr.smithi153.asomxr (mgr.14195) 1699 : cluster 0 pgmap v1418: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:23.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:23 smithi153 bash[14340]: cluster 2024-03-12T02:56:22.189373+0000 mgr.smithi153.asomxr (mgr.14195) 1700 : cluster 0 pgmap v1419: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:23.648 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:56:23.648 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:56:23.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:23 smithi163 bash[22347]: cluster 2024-03-12T02:56:22.189373+0000 mgr.smithi153.asomxr (mgr.14195) 1700 : cluster 0 pgmap v1419: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:24.442 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:56:24.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:24 smithi163 bash[22347]: audit 2024-03-12T02:56:23.634001+0000 mgr.smithi153.asomxr (mgr.14195) 1701 : audit 0 from='client.15434 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:24.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:24 smithi163 bash[22347]: cluster 2024-03-12T02:56:24.190001+0000 mgr.smithi153.asomxr (mgr.14195) 1702 : cluster 0 pgmap v1420: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:24.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:24 smithi153 bash[14340]: audit 2024-03-12T02:56:23.634001+0000 mgr.smithi153.asomxr (mgr.14195) 1701 : audit 0 from='client.15434 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:24.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:24 smithi153 bash[14340]: cluster 2024-03-12T02:56:24.190001+0000 mgr.smithi153.asomxr (mgr.14195) 1702 : cluster 0 pgmap v1420: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:25.443 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:56:27.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:27 smithi153 bash[14340]: cluster 2024-03-12T02:56:26.190633+0000 mgr.smithi153.asomxr (mgr.14195) 1703 : cluster 0 pgmap v1421: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:27.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:27 smithi163 bash[22347]: cluster 2024-03-12T02:56:26.190633+0000 mgr.smithi153.asomxr (mgr.14195) 1703 : cluster 0 pgmap v1421: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:29.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:29 smithi153 bash[14340]: cluster 2024-03-12T02:56:28.191151+0000 mgr.smithi153.asomxr (mgr.14195) 1704 : cluster 0 pgmap v1422: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:29.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:29 smithi163 bash[22347]: cluster 2024-03-12T02:56:28.191151+0000 mgr.smithi153.asomxr (mgr.14195) 1704 : cluster 0 pgmap v1422: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:30.199 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:56:31.505 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:31 smithi153 bash[14340]: cluster 2024-03-12T02:56:30.191852+0000 mgr.smithi153.asomxr (mgr.14195) 1705 : cluster 0 pgmap v1423: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:31.505 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:31 smithi153 bash[14340]: audit 2024-03-12T02:56:30.449468+0000 mon.smithi153 (mon.0) 1216 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:56:31.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:31 smithi163 bash[22347]: cluster 2024-03-12T02:56:30.191852+0000 mgr.smithi153.asomxr (mgr.14195) 1705 : cluster 0 pgmap v1423: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:31.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:31 smithi163 bash[22347]: audit 2024-03-12T02:56:30.449468+0000 mon.smithi153 (mon.0) 1216 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:56:33.387 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:56:33.387 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:56:33.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:33 smithi153 bash[14340]: cluster 2024-03-12T02:56:32.192526+0000 mgr.smithi153.asomxr (mgr.14195) 1706 : cluster 0 pgmap v1424: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:33.621 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:33 smithi163 bash[22347]: cluster 2024-03-12T02:56:32.192526+0000 mgr.smithi153.asomxr (mgr.14195) 1706 : cluster 0 pgmap v1424: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:34.225 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:56:34.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:34 smithi163 bash[22347]: audit 2024-03-12T02:56:33.376297+0000 mgr.smithi153.asomxr (mgr.14195) 1707 : audit 0 from='client.15438 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:34.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:34 smithi163 bash[22347]: cluster 2024-03-12T02:56:34.193095+0000 mgr.smithi153.asomxr (mgr.14195) 1708 : cluster 0 pgmap v1425: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:34.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:34 smithi153 bash[14340]: audit 2024-03-12T02:56:33.376297+0000 mgr.smithi153.asomxr (mgr.14195) 1707 : audit 0 from='client.15438 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:34.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:34 smithi153 bash[14340]: cluster 2024-03-12T02:56:34.193095+0000 mgr.smithi153.asomxr (mgr.14195) 1708 : cluster 0 pgmap v1425: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:35.226 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:56:37.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:37 smithi153 bash[14340]: cluster 2024-03-12T02:56:36.193723+0000 mgr.smithi153.asomxr (mgr.14195) 1709 : cluster 0 pgmap v1426: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:37.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:37 smithi163 bash[22347]: cluster 2024-03-12T02:56:36.193723+0000 mgr.smithi153.asomxr (mgr.14195) 1709 : cluster 0 pgmap v1426: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:39.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:39 smithi153 bash[14340]: cluster 2024-03-12T02:56:38.194360+0000 mgr.smithi153.asomxr (mgr.14195) 1710 : cluster 0 pgmap v1427: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:39.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:39 smithi163 bash[22347]: cluster 2024-03-12T02:56:38.194360+0000 mgr.smithi153.asomxr (mgr.14195) 1710 : cluster 0 pgmap v1427: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:39.976 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:56:41.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:41 smithi153 bash[14340]: cluster 2024-03-12T02:56:40.195000+0000 mgr.smithi153.asomxr (mgr.14195) 1711 : cluster 0 pgmap v1428: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:41.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:41 smithi163 bash[22347]: cluster 2024-03-12T02:56:40.195000+0000 mgr.smithi153.asomxr (mgr.14195) 1711 : cluster 0 pgmap v1428: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:43.271 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:56:43.271 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:56:43.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:43 smithi153 bash[14340]: cluster 2024-03-12T02:56:42.195563+0000 mgr.smithi153.asomxr (mgr.14195) 1712 : cluster 0 pgmap v1429: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:43.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:43 smithi163 bash[22347]: cluster 2024-03-12T02:56:42.195563+0000 mgr.smithi153.asomxr (mgr.14195) 1712 : cluster 0 pgmap v1429: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:44.072 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:56:44.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:44 smithi153 bash[14340]: audit 2024-03-12T02:56:43.255343+0000 mgr.smithi153.asomxr (mgr.14195) 1713 : audit 0 from='client.15442 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:44.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:44 smithi163 bash[22347]: audit 2024-03-12T02:56:43.255343+0000 mgr.smithi153.asomxr (mgr.14195) 1713 : audit 0 from='client.15442 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:45.072 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:56:45.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:45 smithi153 bash[14340]: cluster 2024-03-12T02:56:44.196182+0000 mgr.smithi153.asomxr (mgr.14195) 1714 : cluster 0 pgmap v1430: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:45.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:45 smithi163 bash[22347]: cluster 2024-03-12T02:56:44.196182+0000 mgr.smithi153.asomxr (mgr.14195) 1714 : cluster 0 pgmap v1430: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:46.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:46 smithi153 bash[14340]: audit 2024-03-12T02:56:45.449901+0000 mon.smithi153 (mon.0) 1217 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:56:46.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:46 smithi163 bash[22347]: audit 2024-03-12T02:56:45.449901+0000 mon.smithi153 (mon.0) 1217 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:56:47.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:47 smithi153 bash[14340]: cluster 2024-03-12T02:56:46.196799+0000 mgr.smithi153.asomxr (mgr.14195) 1715 : cluster 0 pgmap v1431: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:47.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:47 smithi163 bash[22347]: cluster 2024-03-12T02:56:46.196799+0000 mgr.smithi153.asomxr (mgr.14195) 1715 : cluster 0 pgmap v1431: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:49.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:49 smithi153 bash[14340]: cluster 2024-03-12T02:56:48.197474+0000 mgr.smithi153.asomxr (mgr.14195) 1716 : cluster 0 pgmap v1432: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:49.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:49 smithi163 bash[22347]: cluster 2024-03-12T02:56:48.197474+0000 mgr.smithi153.asomxr (mgr.14195) 1716 : cluster 0 pgmap v1432: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:49.825 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:56:51.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:51 smithi153 bash[14340]: cluster 2024-03-12T02:56:50.198139+0000 mgr.smithi153.asomxr (mgr.14195) 1717 : cluster 0 pgmap v1433: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:51.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:51 smithi163 bash[22347]: cluster 2024-03-12T02:56:50.198139+0000 mgr.smithi153.asomxr (mgr.14195) 1717 : cluster 0 pgmap v1433: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:53.231 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:56:53.231 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:56:53.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:53 smithi153 bash[14340]: cluster 2024-03-12T02:56:52.198757+0000 mgr.smithi153.asomxr (mgr.14195) 1718 : cluster 0 pgmap v1434: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:53.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:53 smithi163 bash[22347]: cluster 2024-03-12T02:56:52.198757+0000 mgr.smithi153.asomxr (mgr.14195) 1718 : cluster 0 pgmap v1434: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:53.989 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:56:54.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:54 smithi163 bash[22347]: audit 2024-03-12T02:56:53.215098+0000 mgr.smithi153.asomxr (mgr.14195) 1719 : audit 0 from='client.15446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:54.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:54 smithi153 bash[14340]: audit 2024-03-12T02:56:53.215098+0000 mgr.smithi153.asomxr (mgr.14195) 1719 : audit 0 from='client.15446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:56:54.991 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:56:55.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:55 smithi153 bash[14340]: cluster 2024-03-12T02:56:54.199355+0000 mgr.smithi153.asomxr (mgr.14195) 1720 : cluster 0 pgmap v1435: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:55.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:55 smithi163 bash[22347]: cluster 2024-03-12T02:56:54.199355+0000 mgr.smithi153.asomxr (mgr.14195) 1720 : cluster 0 pgmap v1435: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:57.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:57 smithi153 bash[14340]: cluster 2024-03-12T02:56:56.199917+0000 mgr.smithi153.asomxr (mgr.14195) 1721 : cluster 0 pgmap v1436: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:57.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:57 smithi163 bash[22347]: cluster 2024-03-12T02:56:56.199917+0000 mgr.smithi153.asomxr (mgr.14195) 1721 : cluster 0 pgmap v1436: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:59.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:56:59 smithi163 bash[22347]: cluster 2024-03-12T02:56:58.200475+0000 mgr.smithi153.asomxr (mgr.14195) 1722 : cluster 0 pgmap v1437: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:59.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:56:59 smithi153 bash[14340]: cluster 2024-03-12T02:56:58.200475+0000 mgr.smithi153.asomxr (mgr.14195) 1722 : cluster 0 pgmap v1437: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:56:59.748 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:57:00.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:00 smithi153 bash[14340]: audit 2024-03-12T02:57:00.286170+0000 mon.smithi153 (mon.0) 1218 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:57:00.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:00 smithi163 bash[22347]: audit 2024-03-12T02:57:00.286170+0000 mon.smithi153 (mon.0) 1218 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:57:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:01 smithi153 bash[14340]: cluster 2024-03-12T02:57:00.201139+0000 mgr.smithi153.asomxr (mgr.14195) 1723 : cluster 0 pgmap v1438: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:01 smithi153 bash[14340]: audit 2024-03-12T02:57:00.450090+0000 mon.smithi153 (mon.0) 1219 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:57:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:01 smithi153 bash[14340]: audit 2024-03-12T02:57:00.880647+0000 mon.smithi153 (mon.0) 1220 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:57:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:01 smithi153 bash[14340]: audit 2024-03-12T02:57:00.882215+0000 mon.smithi153 (mon.0) 1221 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:57:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:01 smithi153 bash[14340]: audit 2024-03-12T02:57:00.890116+0000 mon.smithi153 (mon.0) 1222 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:57:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:01 smithi153 bash[14340]: audit 2024-03-12T02:57:00.894710+0000 mon.smithi153 (mon.0) 1223 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:57:01.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:01 smithi163 bash[22347]: cluster 2024-03-12T02:57:00.201139+0000 mgr.smithi153.asomxr (mgr.14195) 1723 : cluster 0 pgmap v1438: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:01.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:01 smithi163 bash[22347]: audit 2024-03-12T02:57:00.450090+0000 mon.smithi153 (mon.0) 1219 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:57:01.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:01 smithi163 bash[22347]: audit 2024-03-12T02:57:00.880647+0000 mon.smithi153 (mon.0) 1220 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:57:01.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:01 smithi163 bash[22347]: audit 2024-03-12T02:57:00.882215+0000 mon.smithi153 (mon.0) 1221 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:57:01.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:01 smithi163 bash[22347]: audit 2024-03-12T02:57:00.890116+0000 mon.smithi153 (mon.0) 1222 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:57:01.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:01 smithi163 bash[22347]: audit 2024-03-12T02:57:00.894710+0000 mon.smithi153 (mon.0) 1223 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:57:02.584 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:02 smithi153 bash[14340]: cluster 2024-03-12T02:57:00.884811+0000 mgr.smithi153.asomxr (mgr.14195) 1724 : cluster 0 pgmap v1439: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:02.584 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:02 smithi153 bash[14340]: cluster 2024-03-12T02:57:00.885324+0000 mgr.smithi153.asomxr (mgr.14195) 1725 : cluster 0 pgmap v1440: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:02.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:02 smithi163 bash[22347]: cluster 2024-03-12T02:57:00.884811+0000 mgr.smithi153.asomxr (mgr.14195) 1724 : cluster 0 pgmap v1439: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:02.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:02 smithi163 bash[22347]: cluster 2024-03-12T02:57:00.885324+0000 mgr.smithi153.asomxr (mgr.14195) 1725 : cluster 0 pgmap v1440: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:03.005 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:57:03.005 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:57:03.784 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:57:04.624 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:04 smithi153 bash[14340]: cluster 2024-03-12T02:57:02.886026+0000 mgr.smithi153.asomxr (mgr.14195) 1726 : cluster 0 pgmap v1441: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:04.624 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:04 smithi153 bash[14340]: audit 2024-03-12T02:57:02.988435+0000 mgr.smithi153.asomxr (mgr.14195) 1727 : audit 0 from='client.15450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:04.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:04 smithi163 bash[22347]: cluster 2024-03-12T02:57:02.886026+0000 mgr.smithi153.asomxr (mgr.14195) 1726 : cluster 0 pgmap v1441: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:04.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:04 smithi163 bash[22347]: audit 2024-03-12T02:57:02.988435+0000 mgr.smithi153.asomxr (mgr.14195) 1727 : audit 0 from='client.15450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:04.785 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:57:06.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:06 smithi163 bash[22347]: cluster 2024-03-12T02:57:04.886631+0000 mgr.smithi153.asomxr (mgr.14195) 1728 : cluster 0 pgmap v1442: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:06.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:06 smithi153 bash[14340]: cluster 2024-03-12T02:57:04.886631+0000 mgr.smithi153.asomxr (mgr.14195) 1728 : cluster 0 pgmap v1442: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:08.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:08 smithi163 bash[22347]: cluster 2024-03-12T02:57:06.887244+0000 mgr.smithi153.asomxr (mgr.14195) 1729 : cluster 0 pgmap v1443: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:08.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:08 smithi153 bash[14340]: cluster 2024-03-12T02:57:06.887244+0000 mgr.smithi153.asomxr (mgr.14195) 1729 : cluster 0 pgmap v1443: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:09.533 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:57:09.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:09 smithi163 bash[22347]: cluster 2024-03-12T02:57:08.887893+0000 mgr.smithi153.asomxr (mgr.14195) 1730 : cluster 0 pgmap v1444: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:09.733 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:09 smithi153 bash[14340]: cluster 2024-03-12T02:57:08.887893+0000 mgr.smithi153.asomxr (mgr.14195) 1730 : cluster 0 pgmap v1444: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:12.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:11 smithi163 bash[22347]: cluster 2024-03-12T02:57:10.888510+0000 mgr.smithi153.asomxr (mgr.14195) 1731 : cluster 0 pgmap v1445: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:12.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:11 smithi153 bash[14340]: cluster 2024-03-12T02:57:10.888510+0000 mgr.smithi153.asomxr (mgr.14195) 1731 : cluster 0 pgmap v1445: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:12.876 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:57:12.876 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:57:13.754 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:57:14.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:13 smithi153 bash[14340]: audit 2024-03-12T02:57:12.860192+0000 mgr.smithi153.asomxr (mgr.14195) 1732 : audit 0 from='client.15454 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:14.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:13 smithi153 bash[14340]: cluster 2024-03-12T02:57:12.888968+0000 mgr.smithi153.asomxr (mgr.14195) 1733 : cluster 0 pgmap v1446: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:14.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:13 smithi163 bash[22347]: audit 2024-03-12T02:57:12.860192+0000 mgr.smithi153.asomxr (mgr.14195) 1732 : audit 0 from='client.15454 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:14.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:13 smithi163 bash[22347]: cluster 2024-03-12T02:57:12.888968+0000 mgr.smithi153.asomxr (mgr.14195) 1733 : cluster 0 pgmap v1446: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:14.755 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:57:16.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:15 smithi163 bash[22347]: cluster 2024-03-12T02:57:14.889388+0000 mgr.smithi153.asomxr (mgr.14195) 1734 : cluster 0 pgmap v1447: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:16.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:15 smithi163 bash[22347]: audit 2024-03-12T02:57:15.450562+0000 mon.smithi153 (mon.0) 1224 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:57:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:15 smithi153 bash[14340]: cluster 2024-03-12T02:57:14.889388+0000 mgr.smithi153.asomxr (mgr.14195) 1734 : cluster 0 pgmap v1447: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:15 smithi153 bash[14340]: audit 2024-03-12T02:57:15.450562+0000 mon.smithi153 (mon.0) 1224 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:57:18.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:17 smithi163 bash[22347]: cluster 2024-03-12T02:57:16.889957+0000 mgr.smithi153.asomxr (mgr.14195) 1735 : cluster 0 pgmap v1448: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:18.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:17 smithi153 bash[14340]: cluster 2024-03-12T02:57:16.889957+0000 mgr.smithi153.asomxr (mgr.14195) 1735 : cluster 0 pgmap v1448: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:19.508 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:57:19.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:19 smithi163 bash[22347]: cluster 2024-03-12T02:57:18.890579+0000 mgr.smithi153.asomxr (mgr.14195) 1736 : cluster 0 pgmap v1449: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:19.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:19 smithi153 bash[14340]: cluster 2024-03-12T02:57:18.890579+0000 mgr.smithi153.asomxr (mgr.14195) 1736 : cluster 0 pgmap v1449: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:22.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:21 smithi163 bash[22347]: cluster 2024-03-12T02:57:20.891188+0000 mgr.smithi153.asomxr (mgr.14195) 1737 : cluster 0 pgmap v1450: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:22.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:21 smithi153 bash[14340]: cluster 2024-03-12T02:57:20.891188+0000 mgr.smithi153.asomxr (mgr.14195) 1737 : cluster 0 pgmap v1450: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:22.848 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:57:22.848 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:57:23.669 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:57:24.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:23 smithi163 bash[22347]: audit 2024-03-12T02:57:22.835017+0000 mgr.smithi153.asomxr (mgr.14195) 1738 : audit 0 from='client.15458 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:24.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:23 smithi163 bash[22347]: cluster 2024-03-12T02:57:22.891758+0000 mgr.smithi153.asomxr (mgr.14195) 1739 : cluster 0 pgmap v1451: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:24.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:23 smithi153 bash[14340]: audit 2024-03-12T02:57:22.835017+0000 mgr.smithi153.asomxr (mgr.14195) 1738 : audit 0 from='client.15458 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:24.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:23 smithi153 bash[14340]: cluster 2024-03-12T02:57:22.891758+0000 mgr.smithi153.asomxr (mgr.14195) 1739 : cluster 0 pgmap v1451: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:24.670 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:57:26.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:25 smithi163 bash[22347]: cluster 2024-03-12T02:57:24.892478+0000 mgr.smithi153.asomxr (mgr.14195) 1740 : cluster 0 pgmap v1452: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:26.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:25 smithi153 bash[14340]: cluster 2024-03-12T02:57:24.892478+0000 mgr.smithi153.asomxr (mgr.14195) 1740 : cluster 0 pgmap v1452: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:28.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:27 smithi163 bash[22347]: cluster 2024-03-12T02:57:26.893126+0000 mgr.smithi153.asomxr (mgr.14195) 1741 : cluster 0 pgmap v1453: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:28.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:27 smithi153 bash[14340]: cluster 2024-03-12T02:57:26.893126+0000 mgr.smithi153.asomxr (mgr.14195) 1741 : cluster 0 pgmap v1453: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:29.424 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:57:29.726 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:29 smithi163 bash[22347]: cluster 2024-03-12T02:57:28.893780+0000 mgr.smithi153.asomxr (mgr.14195) 1742 : cluster 0 pgmap v1454: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:29.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:29 smithi153 bash[14340]: cluster 2024-03-12T02:57:28.893780+0000 mgr.smithi153.asomxr (mgr.14195) 1742 : cluster 0 pgmap v1454: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:30.811 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:30 smithi153 bash[14340]: audit 2024-03-12T02:57:30.450401+0000 mon.smithi153 (mon.0) 1225 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:57:30.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:30 smithi163 bash[22347]: audit 2024-03-12T02:57:30.450401+0000 mon.smithi153 (mon.0) 1225 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:57:31.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:31 smithi153 bash[14340]: cluster 2024-03-12T02:57:30.894433+0000 mgr.smithi153.asomxr (mgr.14195) 1743 : cluster 0 pgmap v1455: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:31.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:31 smithi163 bash[22347]: cluster 2024-03-12T02:57:30.894433+0000 mgr.smithi153.asomxr (mgr.14195) 1743 : cluster 0 pgmap v1455: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:32.760 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:57:32.760 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:57:33.501 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:57:34.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:33 smithi163 bash[22347]: audit 2024-03-12T02:57:32.749501+0000 mgr.smithi153.asomxr (mgr.14195) 1744 : audit 0 from='client.15462 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:34.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:33 smithi163 bash[22347]: cluster 2024-03-12T02:57:32.895027+0000 mgr.smithi153.asomxr (mgr.14195) 1745 : cluster 0 pgmap v1456: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:34.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:33 smithi153 bash[14340]: audit 2024-03-12T02:57:32.749501+0000 mgr.smithi153.asomxr (mgr.14195) 1744 : audit 0 from='client.15462 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:34.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:33 smithi153 bash[14340]: cluster 2024-03-12T02:57:32.895027+0000 mgr.smithi153.asomxr (mgr.14195) 1745 : cluster 0 pgmap v1456: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:34.502 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:57:36.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:35 smithi163 bash[22347]: cluster 2024-03-12T02:57:34.895645+0000 mgr.smithi153.asomxr (mgr.14195) 1746 : cluster 0 pgmap v1457: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:36.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:35 smithi153 bash[14340]: cluster 2024-03-12T02:57:34.895645+0000 mgr.smithi153.asomxr (mgr.14195) 1746 : cluster 0 pgmap v1457: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:38.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:37 smithi163 bash[22347]: cluster 2024-03-12T02:57:36.896273+0000 mgr.smithi153.asomxr (mgr.14195) 1747 : cluster 0 pgmap v1458: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:38.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:37 smithi153 bash[14340]: cluster 2024-03-12T02:57:36.896273+0000 mgr.smithi153.asomxr (mgr.14195) 1747 : cluster 0 pgmap v1458: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:39.262 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:57:39.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:39 smithi153 bash[14340]: cluster 2024-03-12T02:57:38.896882+0000 mgr.smithi153.asomxr (mgr.14195) 1748 : cluster 0 pgmap v1459: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:39.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:39 smithi163 bash[22347]: cluster 2024-03-12T02:57:38.896882+0000 mgr.smithi153.asomxr (mgr.14195) 1748 : cluster 0 pgmap v1459: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:42.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:41 smithi163 bash[22347]: cluster 2024-03-12T02:57:40.897557+0000 mgr.smithi153.asomxr (mgr.14195) 1749 : cluster 0 pgmap v1460: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:42.261 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:41 smithi153 bash[14340]: cluster 2024-03-12T02:57:40.897557+0000 mgr.smithi153.asomxr (mgr.14195) 1749 : cluster 0 pgmap v1460: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:42.714 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:57:42.715 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:57:43.445 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:57:44.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:43 smithi163 bash[22347]: audit 2024-03-12T02:57:42.700339+0000 mgr.smithi153.asomxr (mgr.14195) 1750 : audit 0 from='client.15466 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:44.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:43 smithi163 bash[22347]: cluster 2024-03-12T02:57:42.898188+0000 mgr.smithi153.asomxr (mgr.14195) 1751 : cluster 0 pgmap v1461: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:44.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:43 smithi153 bash[14340]: audit 2024-03-12T02:57:42.700339+0000 mgr.smithi153.asomxr (mgr.14195) 1750 : audit 0 from='client.15466 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:44.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:43 smithi153 bash[14340]: cluster 2024-03-12T02:57:42.898188+0000 mgr.smithi153.asomxr (mgr.14195) 1751 : cluster 0 pgmap v1461: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:44.445 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:57:46.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:45 smithi163 bash[22347]: cluster 2024-03-12T02:57:44.898800+0000 mgr.smithi153.asomxr (mgr.14195) 1752 : cluster 0 pgmap v1462: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:46.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:45 smithi163 bash[22347]: audit 2024-03-12T02:57:45.451511+0000 mon.smithi153 (mon.0) 1226 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:57:46.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:45 smithi153 bash[14340]: cluster 2024-03-12T02:57:44.898800+0000 mgr.smithi153.asomxr (mgr.14195) 1752 : cluster 0 pgmap v1462: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:46.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:45 smithi153 bash[14340]: audit 2024-03-12T02:57:45.451511+0000 mon.smithi153 (mon.0) 1226 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:57:48.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:47 smithi163 bash[22347]: cluster 2024-03-12T02:57:46.899460+0000 mgr.smithi153.asomxr (mgr.14195) 1753 : cluster 0 pgmap v1463: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:48.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:47 smithi153 bash[14340]: cluster 2024-03-12T02:57:46.899460+0000 mgr.smithi153.asomxr (mgr.14195) 1753 : cluster 0 pgmap v1463: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:49.197 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:57:49.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:49 smithi163 bash[22347]: cluster 2024-03-12T02:57:48.900087+0000 mgr.smithi153.asomxr (mgr.14195) 1754 : cluster 0 pgmap v1464: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:49.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:49 smithi153 bash[14340]: cluster 2024-03-12T02:57:48.900087+0000 mgr.smithi153.asomxr (mgr.14195) 1754 : cluster 0 pgmap v1464: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:52.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:51 smithi163 bash[22347]: cluster 2024-03-12T02:57:50.900712+0000 mgr.smithi153.asomxr (mgr.14195) 1755 : cluster 0 pgmap v1465: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:52.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:51 smithi153 bash[14340]: cluster 2024-03-12T02:57:50.900712+0000 mgr.smithi153.asomxr (mgr.14195) 1755 : cluster 0 pgmap v1465: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:52.441 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:57:52.441 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:57:53.301 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:57:54.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:53 smithi163 bash[22347]: audit 2024-03-12T02:57:52.426106+0000 mgr.smithi153.asomxr (mgr.14195) 1756 : audit 0 from='client.15470 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:54.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:53 smithi163 bash[22347]: cluster 2024-03-12T02:57:52.901147+0000 mgr.smithi153.asomxr (mgr.14195) 1757 : cluster 0 pgmap v1466: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:54.302 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:57:54.315 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:53 smithi153 bash[14340]: audit 2024-03-12T02:57:52.426106+0000 mgr.smithi153.asomxr (mgr.14195) 1756 : audit 0 from='client.15470 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:57:54.315 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:53 smithi153 bash[14340]: cluster 2024-03-12T02:57:52.901147+0000 mgr.smithi153.asomxr (mgr.14195) 1757 : cluster 0 pgmap v1466: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:56.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:55 smithi163 bash[22347]: cluster 2024-03-12T02:57:54.901657+0000 mgr.smithi153.asomxr (mgr.14195) 1758 : cluster 0 pgmap v1467: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:56.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:55 smithi153 bash[14340]: cluster 2024-03-12T02:57:54.901657+0000 mgr.smithi153.asomxr (mgr.14195) 1758 : cluster 0 pgmap v1467: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:58.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:57 smithi163 bash[22347]: cluster 2024-03-12T02:57:56.902229+0000 mgr.smithi153.asomxr (mgr.14195) 1759 : cluster 0 pgmap v1468: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:57 smithi153 bash[14340]: cluster 2024-03-12T02:57:56.902229+0000 mgr.smithi153.asomxr (mgr.14195) 1759 : cluster 0 pgmap v1468: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:59.054 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:57:59.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:57:59 smithi163 bash[22347]: cluster 2024-03-12T02:57:58.902984+0000 mgr.smithi153.asomxr (mgr.14195) 1760 : cluster 0 pgmap v1469: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:57:59.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:57:59 smithi153 bash[14340]: cluster 2024-03-12T02:57:58.902984+0000 mgr.smithi153.asomxr (mgr.14195) 1760 : cluster 0 pgmap v1469: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:00.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:00 smithi153 bash[14340]: audit 2024-03-12T02:58:00.451744+0000 mon.smithi153 (mon.0) 1227 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:58:00.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:00 smithi163 bash[22347]: audit 2024-03-12T02:58:00.451744+0000 mon.smithi153 (mon.0) 1227 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:58:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:01 smithi153 bash[14340]: cluster 2024-03-12T02:58:00.903534+0000 mgr.smithi153.asomxr (mgr.14195) 1761 : cluster 0 pgmap v1470: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:01 smithi153 bash[14340]: audit 2024-03-12T02:58:00.986264+0000 mon.smithi153 (mon.0) 1228 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:58:01.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:01 smithi163 bash[22347]: cluster 2024-03-12T02:58:00.903534+0000 mgr.smithi153.asomxr (mgr.14195) 1761 : cluster 0 pgmap v1470: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:01.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:01 smithi163 bash[22347]: audit 2024-03-12T02:58:00.986264+0000 mon.smithi153 (mon.0) 1228 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:58:02.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:02 smithi153 bash[14340]: audit 2024-03-12T02:58:01.511809+0000 mon.smithi153 (mon.0) 1229 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:58:02.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:02 smithi153 bash[14340]: audit 2024-03-12T02:58:01.517371+0000 mon.smithi153 (mon.0) 1230 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:58:02.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:02 smithi153 bash[14340]: cluster 2024-03-12T02:58:01.519056+0000 mgr.smithi153.asomxr (mgr.14195) 1762 : cluster 0 pgmap v1471: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:02.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:02 smithi153 bash[14340]: cluster 2024-03-12T02:58:01.519207+0000 mgr.smithi153.asomxr (mgr.14195) 1763 : cluster 0 pgmap v1472: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:02.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:02 smithi153 bash[14340]: audit 2024-03-12T02:58:01.522786+0000 mon.smithi153 (mon.0) 1231 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:58:02.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:02 smithi153 bash[14340]: audit 2024-03-12T02:58:01.526010+0000 mon.smithi153 (mon.0) 1232 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:58:02.748 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:58:02.748 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:58:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:02 smithi163 bash[22347]: audit 2024-03-12T02:58:01.511809+0000 mon.smithi153 (mon.0) 1229 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:58:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:02 smithi163 bash[22347]: audit 2024-03-12T02:58:01.517371+0000 mon.smithi153 (mon.0) 1230 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:58:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:02 smithi163 bash[22347]: cluster 2024-03-12T02:58:01.519056+0000 mgr.smithi153.asomxr (mgr.14195) 1762 : cluster 0 pgmap v1471: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:02 smithi163 bash[22347]: cluster 2024-03-12T02:58:01.519207+0000 mgr.smithi153.asomxr (mgr.14195) 1763 : cluster 0 pgmap v1472: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:02 smithi163 bash[22347]: audit 2024-03-12T02:58:01.522786+0000 mon.smithi153 (mon.0) 1231 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:58:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:02 smithi163 bash[22347]: audit 2024-03-12T02:58:01.526010+0000 mon.smithi153 (mon.0) 1232 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:58:03.605 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:58:03.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:03 smithi153 bash[14340]: audit 2024-03-12T02:58:02.737687+0000 mgr.smithi153.asomxr (mgr.14195) 1764 : audit 0 from='client.15474 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:03.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:03 smithi163 bash[22347]: audit 2024-03-12T02:58:02.737687+0000 mgr.smithi153.asomxr (mgr.14195) 1764 : audit 0 from='client.15474 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:04.606 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:58:04.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:04 smithi153 bash[14340]: cluster 2024-03-12T02:58:03.519847+0000 mgr.smithi153.asomxr (mgr.14195) 1765 : cluster 0 pgmap v1473: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:04.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:04 smithi163 bash[22347]: cluster 2024-03-12T02:58:03.519847+0000 mgr.smithi153.asomxr (mgr.14195) 1765 : cluster 0 pgmap v1473: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:06.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:06 smithi153 bash[14340]: cluster 2024-03-12T02:58:05.520447+0000 mgr.smithi153.asomxr (mgr.14195) 1766 : cluster 0 pgmap v1474: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:06.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:06 smithi163 bash[22347]: cluster 2024-03-12T02:58:05.520447+0000 mgr.smithi153.asomxr (mgr.14195) 1766 : cluster 0 pgmap v1474: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:08.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:08 smithi153 bash[14340]: cluster 2024-03-12T02:58:07.521009+0000 mgr.smithi153.asomxr (mgr.14195) 1767 : cluster 0 pgmap v1475: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:08.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:08 smithi163 bash[22347]: cluster 2024-03-12T02:58:07.521009+0000 mgr.smithi153.asomxr (mgr.14195) 1767 : cluster 0 pgmap v1475: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:09.362 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:58:10.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:10 smithi153 bash[14340]: cluster 2024-03-12T02:58:09.521650+0000 mgr.smithi153.asomxr (mgr.14195) 1768 : cluster 0 pgmap v1476: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:10.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:10 smithi163 bash[22347]: cluster 2024-03-12T02:58:09.521650+0000 mgr.smithi153.asomxr (mgr.14195) 1768 : cluster 0 pgmap v1476: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:12.814 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:58:12.815 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:58:12.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:12 smithi153 bash[14340]: cluster 2024-03-12T02:58:11.522261+0000 mgr.smithi153.asomxr (mgr.14195) 1769 : cluster 0 pgmap v1477: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:12.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:12 smithi163 bash[22347]: cluster 2024-03-12T02:58:11.522261+0000 mgr.smithi153.asomxr (mgr.14195) 1769 : cluster 0 pgmap v1477: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:13.656 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:58:13.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:13 smithi153 bash[14340]: audit 2024-03-12T02:58:12.799691+0000 mgr.smithi153.asomxr (mgr.14195) 1770 : audit 0 from='client.15478 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:13.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:13 smithi163 bash[22347]: audit 2024-03-12T02:58:12.799691+0000 mgr.smithi153.asomxr (mgr.14195) 1770 : audit 0 from='client.15478 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:14.657 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:58:14.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:14 smithi153 bash[14340]: cluster 2024-03-12T02:58:13.522862+0000 mgr.smithi153.asomxr (mgr.14195) 1771 : cluster 0 pgmap v1478: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:14.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:14 smithi163 bash[22347]: cluster 2024-03-12T02:58:13.522862+0000 mgr.smithi153.asomxr (mgr.14195) 1771 : cluster 0 pgmap v1478: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:15.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:15 smithi153 bash[14340]: audit 2024-03-12T02:58:15.452185+0000 mon.smithi153 (mon.0) 1233 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:58:15.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:15 smithi163 bash[22347]: audit 2024-03-12T02:58:15.452185+0000 mon.smithi153 (mon.0) 1233 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:58:16.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:16 smithi153 bash[14340]: cluster 2024-03-12T02:58:15.523473+0000 mgr.smithi153.asomxr (mgr.14195) 1772 : cluster 0 pgmap v1479: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:16.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:16 smithi163 bash[22347]: cluster 2024-03-12T02:58:15.523473+0000 mgr.smithi153.asomxr (mgr.14195) 1772 : cluster 0 pgmap v1479: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:18.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:18 smithi153 bash[14340]: cluster 2024-03-12T02:58:17.524108+0000 mgr.smithi153.asomxr (mgr.14195) 1773 : cluster 0 pgmap v1480: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:18.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:18 smithi163 bash[22347]: cluster 2024-03-12T02:58:17.524108+0000 mgr.smithi153.asomxr (mgr.14195) 1773 : cluster 0 pgmap v1480: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:19.415 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:58:20.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:20 smithi153 bash[14340]: cluster 2024-03-12T02:58:19.524758+0000 mgr.smithi153.asomxr (mgr.14195) 1774 : cluster 0 pgmap v1481: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:20.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:20 smithi163 bash[22347]: cluster 2024-03-12T02:58:19.524758+0000 mgr.smithi153.asomxr (mgr.14195) 1774 : cluster 0 pgmap v1481: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:22.811 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:58:22.811 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:58:22.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:22 smithi153 bash[14340]: cluster 2024-03-12T02:58:21.525167+0000 mgr.smithi153.asomxr (mgr.14195) 1775 : cluster 0 pgmap v1482: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:22.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:22 smithi163 bash[22347]: cluster 2024-03-12T02:58:21.525167+0000 mgr.smithi153.asomxr (mgr.14195) 1775 : cluster 0 pgmap v1482: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:23.669 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:58:23.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:23 smithi163 bash[22347]: audit 2024-03-12T02:58:22.794965+0000 mgr.smithi153.asomxr (mgr.14195) 1776 : audit 0 from='client.15482 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:24.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:23 smithi153 bash[14340]: audit 2024-03-12T02:58:22.794965+0000 mgr.smithi153.asomxr (mgr.14195) 1776 : audit 0 from='client.15482 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:24.670 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:58:24.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:24 smithi163 bash[22347]: cluster 2024-03-12T02:58:23.525776+0000 mgr.smithi153.asomxr (mgr.14195) 1777 : cluster 0 pgmap v1483: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:25.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:24 smithi153 bash[14340]: cluster 2024-03-12T02:58:23.525776+0000 mgr.smithi153.asomxr (mgr.14195) 1777 : cluster 0 pgmap v1483: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:26.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:26 smithi163 bash[22347]: cluster 2024-03-12T02:58:25.526439+0000 mgr.smithi153.asomxr (mgr.14195) 1778 : cluster 0 pgmap v1484: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:27.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:26 smithi153 bash[14340]: cluster 2024-03-12T02:58:25.526439+0000 mgr.smithi153.asomxr (mgr.14195) 1778 : cluster 0 pgmap v1484: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:28.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:28 smithi163 bash[22347]: cluster 2024-03-12T02:58:27.527009+0000 mgr.smithi153.asomxr (mgr.14195) 1779 : cluster 0 pgmap v1485: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:29.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:28 smithi153 bash[14340]: cluster 2024-03-12T02:58:27.527009+0000 mgr.smithi153.asomxr (mgr.14195) 1779 : cluster 0 pgmap v1485: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:29.423 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:58:30.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:30 smithi153 bash[14340]: cluster 2024-03-12T02:58:29.527737+0000 mgr.smithi153.asomxr (mgr.14195) 1780 : cluster 0 pgmap v1486: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:30.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:30 smithi153 bash[14340]: audit 2024-03-12T02:58:30.452403+0000 mon.smithi153 (mon.0) 1234 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:58:30.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:30 smithi163 bash[22347]: cluster 2024-03-12T02:58:29.527737+0000 mgr.smithi153.asomxr (mgr.14195) 1780 : cluster 0 pgmap v1486: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:30.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:30 smithi163 bash[22347]: audit 2024-03-12T02:58:30.452403+0000 mon.smithi153 (mon.0) 1234 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:58:32.731 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:58:32.731 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:58:32.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:32 smithi153 bash[14340]: cluster 2024-03-12T02:58:31.528358+0000 mgr.smithi153.asomxr (mgr.14195) 1781 : cluster 0 pgmap v1487: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:32.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:32 smithi163 bash[22347]: cluster 2024-03-12T02:58:31.528358+0000 mgr.smithi153.asomxr (mgr.14195) 1781 : cluster 0 pgmap v1487: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:33.578 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:58:33.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:33 smithi163 bash[22347]: audit 2024-03-12T02:58:32.717539+0000 mgr.smithi153.asomxr (mgr.14195) 1782 : audit 0 from='client.15486 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:33 smithi153 bash[14340]: audit 2024-03-12T02:58:32.717539+0000 mgr.smithi153.asomxr (mgr.14195) 1782 : audit 0 from='client.15486 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:34.578 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:58:34.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:34 smithi163 bash[22347]: cluster 2024-03-12T02:58:33.529031+0000 mgr.smithi153.asomxr (mgr.14195) 1783 : cluster 0 pgmap v1488: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:35.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:34 smithi153 bash[14340]: cluster 2024-03-12T02:58:33.529031+0000 mgr.smithi153.asomxr (mgr.14195) 1783 : cluster 0 pgmap v1488: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:36.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:36 smithi153 bash[14340]: cluster 2024-03-12T02:58:35.529680+0000 mgr.smithi153.asomxr (mgr.14195) 1784 : cluster 0 pgmap v1489: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:36.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:36 smithi163 bash[22347]: cluster 2024-03-12T02:58:35.529680+0000 mgr.smithi153.asomxr (mgr.14195) 1784 : cluster 0 pgmap v1489: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:38.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:38 smithi163 bash[22347]: cluster 2024-03-12T02:58:37.530339+0000 mgr.smithi153.asomxr (mgr.14195) 1785 : cluster 0 pgmap v1490: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:39.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:38 smithi153 bash[14340]: cluster 2024-03-12T02:58:37.530339+0000 mgr.smithi153.asomxr (mgr.14195) 1785 : cluster 0 pgmap v1490: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:39.333 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:58:40.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:40 smithi163 bash[22347]: cluster 2024-03-12T02:58:39.530959+0000 mgr.smithi153.asomxr (mgr.14195) 1786 : cluster 0 pgmap v1491: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:41.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:40 smithi153 bash[14340]: cluster 2024-03-12T02:58:39.530959+0000 mgr.smithi153.asomxr (mgr.14195) 1786 : cluster 0 pgmap v1491: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:42.464 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:58:42.464 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:58:42.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:42 smithi153 bash[14340]: cluster 2024-03-12T02:58:41.531419+0000 mgr.smithi153.asomxr (mgr.14195) 1787 : cluster 0 pgmap v1492: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:42.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:42 smithi163 bash[22347]: cluster 2024-03-12T02:58:41.531419+0000 mgr.smithi153.asomxr (mgr.14195) 1787 : cluster 0 pgmap v1492: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:43.319 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:58:43.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:43 smithi163 bash[22347]: audit 2024-03-12T02:58:42.453719+0000 mgr.smithi153.asomxr (mgr.14195) 1788 : audit 0 from='client.15490 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:44.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:43 smithi153 bash[14340]: audit 2024-03-12T02:58:42.453719+0000 mgr.smithi153.asomxr (mgr.14195) 1788 : audit 0 from='client.15490 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:44.321 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:58:44.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:44 smithi163 bash[22347]: cluster 2024-03-12T02:58:43.532004+0000 mgr.smithi153.asomxr (mgr.14195) 1789 : cluster 0 pgmap v1493: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:45.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:44 smithi153 bash[14340]: cluster 2024-03-12T02:58:43.532004+0000 mgr.smithi153.asomxr (mgr.14195) 1789 : cluster 0 pgmap v1493: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:45.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:45 smithi163 bash[22347]: audit 2024-03-12T02:58:45.453050+0000 mon.smithi153 (mon.0) 1235 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:58:46.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:45 smithi153 bash[14340]: audit 2024-03-12T02:58:45.453050+0000 mon.smithi153 (mon.0) 1235 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:58:46.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:46 smithi163 bash[22347]: cluster 2024-03-12T02:58:45.532723+0000 mgr.smithi153.asomxr (mgr.14195) 1790 : cluster 0 pgmap v1494: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:47.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:46 smithi153 bash[14340]: cluster 2024-03-12T02:58:45.532723+0000 mgr.smithi153.asomxr (mgr.14195) 1790 : cluster 0 pgmap v1494: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:48.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:48 smithi163 bash[22347]: cluster 2024-03-12T02:58:47.533244+0000 mgr.smithi153.asomxr (mgr.14195) 1791 : cluster 0 pgmap v1495: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:49.073 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:58:49.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:48 smithi153 bash[14340]: cluster 2024-03-12T02:58:47.533244+0000 mgr.smithi153.asomxr (mgr.14195) 1791 : cluster 0 pgmap v1495: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:50.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:50 smithi153 bash[14340]: cluster 2024-03-12T02:58:49.533916+0000 mgr.smithi153.asomxr (mgr.14195) 1792 : cluster 0 pgmap v1496: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:50.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:50 smithi163 bash[22347]: cluster 2024-03-12T02:58:49.533916+0000 mgr.smithi153.asomxr (mgr.14195) 1792 : cluster 0 pgmap v1496: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:52.324 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:58:52.324 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:58:52.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:52 smithi153 bash[14340]: cluster 2024-03-12T02:58:51.534593+0000 mgr.smithi153.asomxr (mgr.14195) 1793 : cluster 0 pgmap v1497: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:52.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:52 smithi153 bash[14340]: audit 2024-03-12T02:58:52.314199+0000 mgr.smithi153.asomxr (mgr.14195) 1794 : audit 0 from='client.15494 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:52.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:52 smithi163 bash[22347]: cluster 2024-03-12T02:58:51.534593+0000 mgr.smithi153.asomxr (mgr.14195) 1793 : cluster 0 pgmap v1497: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:52.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:52 smithi163 bash[22347]: audit 2024-03-12T02:58:52.314199+0000 mgr.smithi153.asomxr (mgr.14195) 1794 : audit 0 from='client.15494 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:58:53.143 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:58:54.145 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:58:54.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:54 smithi163 bash[22347]: cluster 2024-03-12T02:58:53.535231+0000 mgr.smithi153.asomxr (mgr.14195) 1795 : cluster 0 pgmap v1498: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:54.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:54 smithi153 bash[14340]: cluster 2024-03-12T02:58:53.535231+0000 mgr.smithi153.asomxr (mgr.14195) 1795 : cluster 0 pgmap v1498: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:56.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:56 smithi163 bash[22347]: cluster 2024-03-12T02:58:55.535864+0000 mgr.smithi153.asomxr (mgr.14195) 1796 : cluster 0 pgmap v1499: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:57.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:56 smithi153 bash[14340]: cluster 2024-03-12T02:58:55.535864+0000 mgr.smithi153.asomxr (mgr.14195) 1796 : cluster 0 pgmap v1499: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:58.893 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:58:58.922 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:58:58 smithi153 bash[14340]: cluster 2024-03-12T02:58:57.536512+0000 mgr.smithi153.asomxr (mgr.14195) 1797 : cluster 0 pgmap v1500: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:58:58.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:58:58 smithi163 bash[22347]: cluster 2024-03-12T02:58:57.536512+0000 mgr.smithi153.asomxr (mgr.14195) 1797 : cluster 0 pgmap v1500: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:00.865 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:00 smithi153 bash[14340]: cluster 2024-03-12T02:58:59.537129+0000 mgr.smithi153.asomxr (mgr.14195) 1798 : cluster 0 pgmap v1501: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:00.865 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:00 smithi153 bash[14340]: audit 2024-03-12T02:59:00.453279+0000 mon.smithi153 (mon.0) 1236 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:59:00.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:00 smithi163 bash[22347]: cluster 2024-03-12T02:58:59.537129+0000 mgr.smithi153.asomxr (mgr.14195) 1798 : cluster 0 pgmap v1501: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:00.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:00 smithi163 bash[22347]: audit 2024-03-12T02:59:00.453279+0000 mon.smithi153 (mon.0) 1236 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:59:02.257 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:59:02.257 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:59:02.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:02 smithi153 bash[14340]: cluster 2024-03-12T02:59:01.537630+0000 mgr.smithi153.asomxr (mgr.14195) 1799 : cluster 0 pgmap v1502: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:02.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:02 smithi153 bash[14340]: audit 2024-03-12T02:59:01.615439+0000 mon.smithi153 (mon.0) 1237 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:59:02.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:02 smithi153 bash[14340]: audit 2024-03-12T02:59:02.199584+0000 mon.smithi153 (mon.0) 1238 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:59:02.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:02 smithi153 bash[14340]: audit 2024-03-12T02:59:02.201053+0000 mon.smithi153 (mon.0) 1239 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:59:02.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:02 smithi153 bash[14340]: cluster 2024-03-12T02:59:02.203477+0000 mgr.smithi153.asomxr (mgr.14195) 1800 : cluster 0 pgmap v1503: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:02.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:02 smithi153 bash[14340]: audit 2024-03-12T02:59:02.210113+0000 mon.smithi153 (mon.0) 1240 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:59:02.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:02 smithi153 bash[14340]: audit 2024-03-12T02:59:02.214499+0000 mon.smithi153 (mon.0) 1241 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:59:02.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:02 smithi153 bash[14340]: audit 2024-03-12T02:59:02.244568+0000 mgr.smithi153.asomxr (mgr.14195) 1801 : audit 0 from='client.15498 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:02 smithi163 bash[22347]: cluster 2024-03-12T02:59:01.537630+0000 mgr.smithi153.asomxr (mgr.14195) 1799 : cluster 0 pgmap v1502: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:02 smithi163 bash[22347]: audit 2024-03-12T02:59:01.615439+0000 mon.smithi153 (mon.0) 1237 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T02:59:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:02 smithi163 bash[22347]: audit 2024-03-12T02:59:02.199584+0000 mon.smithi153 (mon.0) 1238 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T02:59:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:02 smithi163 bash[22347]: audit 2024-03-12T02:59:02.201053+0000 mon.smithi153 (mon.0) 1239 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T02:59:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:02 smithi163 bash[22347]: cluster 2024-03-12T02:59:02.203477+0000 mgr.smithi153.asomxr (mgr.14195) 1800 : cluster 0 pgmap v1503: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:02 smithi163 bash[22347]: audit 2024-03-12T02:59:02.210113+0000 mon.smithi153 (mon.0) 1240 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T02:59:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:02 smithi163 bash[22347]: audit 2024-03-12T02:59:02.214499+0000 mon.smithi153 (mon.0) 1241 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T02:59:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:02 smithi163 bash[22347]: audit 2024-03-12T02:59:02.244568+0000 mgr.smithi153.asomxr (mgr.14195) 1801 : audit 0 from='client.15498 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:03.054 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:59:04.056 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:59:04.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:04 smithi163 bash[22347]: cluster 2024-03-12T02:59:04.203865+0000 mgr.smithi153.asomxr (mgr.14195) 1802 : cluster 0 pgmap v1504: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:04.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:04 smithi153 bash[14340]: cluster 2024-03-12T02:59:04.203865+0000 mgr.smithi153.asomxr (mgr.14195) 1802 : cluster 0 pgmap v1504: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:07.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:07 smithi153 bash[14340]: cluster 2024-03-12T02:59:06.204469+0000 mgr.smithi153.asomxr (mgr.14195) 1803 : cluster 0 pgmap v1505: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:07.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:07 smithi163 bash[22347]: cluster 2024-03-12T02:59:06.204469+0000 mgr.smithi153.asomxr (mgr.14195) 1803 : cluster 0 pgmap v1505: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:08.807 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:59:09.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:09 smithi153 bash[14340]: cluster 2024-03-12T02:59:08.205123+0000 mgr.smithi153.asomxr (mgr.14195) 1804 : cluster 0 pgmap v1506: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:09.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:09 smithi163 bash[22347]: cluster 2024-03-12T02:59:08.205123+0000 mgr.smithi153.asomxr (mgr.14195) 1804 : cluster 0 pgmap v1506: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:11.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:11 smithi153 bash[14340]: cluster 2024-03-12T02:59:10.205747+0000 mgr.smithi153.asomxr (mgr.14195) 1805 : cluster 0 pgmap v1507: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:11.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:11 smithi163 bash[22347]: cluster 2024-03-12T02:59:10.205747+0000 mgr.smithi153.asomxr (mgr.14195) 1805 : cluster 0 pgmap v1507: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:12.113 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:59:12.113 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:59:12.921 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:59:13.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:13 smithi153 bash[14340]: audit 2024-03-12T02:59:12.102307+0000 mgr.smithi153.asomxr (mgr.14195) 1806 : audit 0 from='client.15502 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:13.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:13 smithi153 bash[14340]: cluster 2024-03-12T02:59:12.206236+0000 mgr.smithi153.asomxr (mgr.14195) 1807 : cluster 0 pgmap v1508: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:13.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:13 smithi163 bash[22347]: audit 2024-03-12T02:59:12.102307+0000 mgr.smithi153.asomxr (mgr.14195) 1806 : audit 0 from='client.15502 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:13.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:13 smithi163 bash[22347]: cluster 2024-03-12T02:59:12.206236+0000 mgr.smithi153.asomxr (mgr.14195) 1807 : cluster 0 pgmap v1508: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:13.922 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:59:14.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:14 smithi163 bash[22347]: cluster 2024-03-12T02:59:14.206812+0000 mgr.smithi153.asomxr (mgr.14195) 1808 : cluster 0 pgmap v1509: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:14.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:14 smithi153 bash[14340]: cluster 2024-03-12T02:59:14.206812+0000 mgr.smithi153.asomxr (mgr.14195) 1808 : cluster 0 pgmap v1509: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:15.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:15 smithi153 bash[14340]: audit 2024-03-12T02:59:15.453594+0000 mon.smithi153 (mon.0) 1242 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:59:15.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:15 smithi163 bash[22347]: audit 2024-03-12T02:59:15.453594+0000 mon.smithi153 (mon.0) 1242 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:59:16.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:16 smithi153 bash[14340]: cluster 2024-03-12T02:59:16.207472+0000 mgr.smithi153.asomxr (mgr.14195) 1809 : cluster 0 pgmap v1510: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:16.900 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:16 smithi163 bash[22347]: cluster 2024-03-12T02:59:16.207472+0000 mgr.smithi153.asomxr (mgr.14195) 1809 : cluster 0 pgmap v1510: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:18.674 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:59:19.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:19 smithi153 bash[14340]: cluster 2024-03-12T02:59:18.208068+0000 mgr.smithi153.asomxr (mgr.14195) 1810 : cluster 0 pgmap v1511: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:19.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:19 smithi163 bash[22347]: cluster 2024-03-12T02:59:18.208068+0000 mgr.smithi153.asomxr (mgr.14195) 1810 : cluster 0 pgmap v1511: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:21.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:21 smithi153 bash[14340]: cluster 2024-03-12T02:59:20.208704+0000 mgr.smithi153.asomxr (mgr.14195) 1811 : cluster 0 pgmap v1512: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:21.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:21 smithi163 bash[22347]: cluster 2024-03-12T02:59:20.208704+0000 mgr.smithi153.asomxr (mgr.14195) 1811 : cluster 0 pgmap v1512: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:21.888 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:59:21.888 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:59:22.718 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:59:23.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:23 smithi153 bash[14340]: audit 2024-03-12T02:59:21.872874+0000 mgr.smithi153.asomxr (mgr.14195) 1812 : audit 0 from='client.15506 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:23.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:23 smithi153 bash[14340]: cluster 2024-03-12T02:59:22.209445+0000 mgr.smithi153.asomxr (mgr.14195) 1813 : cluster 0 pgmap v1513: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:23.718 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:59:23.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:23 smithi163 bash[22347]: audit 2024-03-12T02:59:21.872874+0000 mgr.smithi153.asomxr (mgr.14195) 1812 : audit 0 from='client.15506 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:23.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:23 smithi163 bash[22347]: cluster 2024-03-12T02:59:22.209445+0000 mgr.smithi153.asomxr (mgr.14195) 1813 : cluster 0 pgmap v1513: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:24.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:24 smithi163 bash[22347]: cluster 2024-03-12T02:59:24.210048+0000 mgr.smithi153.asomxr (mgr.14195) 1814 : cluster 0 pgmap v1514: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:24.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:24 smithi153 bash[14340]: cluster 2024-03-12T02:59:24.210048+0000 mgr.smithi153.asomxr (mgr.14195) 1814 : cluster 0 pgmap v1514: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:27.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:27 smithi163 bash[22347]: cluster 2024-03-12T02:59:26.210679+0000 mgr.smithi153.asomxr (mgr.14195) 1815 : cluster 0 pgmap v1515: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:27.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:27 smithi153 bash[14340]: cluster 2024-03-12T02:59:26.210679+0000 mgr.smithi153.asomxr (mgr.14195) 1815 : cluster 0 pgmap v1515: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:28.470 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:59:29.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:29 smithi153 bash[14340]: cluster 2024-03-12T02:59:28.211320+0000 mgr.smithi153.asomxr (mgr.14195) 1816 : cluster 0 pgmap v1516: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:29.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:29 smithi163 bash[22347]: cluster 2024-03-12T02:59:28.211320+0000 mgr.smithi153.asomxr (mgr.14195) 1816 : cluster 0 pgmap v1516: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:31 smithi153 bash[14340]: cluster 2024-03-12T02:59:30.211957+0000 mgr.smithi153.asomxr (mgr.14195) 1817 : cluster 0 pgmap v1517: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:31 smithi153 bash[14340]: audit 2024-03-12T02:59:30.453887+0000 mon.smithi153 (mon.0) 1243 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:59:31.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:31 smithi163 bash[22347]: cluster 2024-03-12T02:59:30.211957+0000 mgr.smithi153.asomxr (mgr.14195) 1817 : cluster 0 pgmap v1517: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:31.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:31 smithi163 bash[22347]: audit 2024-03-12T02:59:30.453887+0000 mon.smithi153 (mon.0) 1243 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:59:31.741 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:59:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:59:32.536 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:59:33.536 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:59:33.550 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:33 smithi153 bash[14340]: audit 2024-03-12T02:59:31.725254+0000 mgr.smithi153.asomxr (mgr.14195) 1818 : audit 0 from='client.15510 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:33.550 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:33 smithi153 bash[14340]: cluster 2024-03-12T02:59:32.212509+0000 mgr.smithi153.asomxr (mgr.14195) 1819 : cluster 0 pgmap v1518: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:33.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:33 smithi163 bash[22347]: audit 2024-03-12T02:59:31.725254+0000 mgr.smithi153.asomxr (mgr.14195) 1818 : audit 0 from='client.15510 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:33.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:33 smithi163 bash[22347]: cluster 2024-03-12T02:59:32.212509+0000 mgr.smithi153.asomxr (mgr.14195) 1819 : cluster 0 pgmap v1518: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:34.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:34 smithi163 bash[22347]: cluster 2024-03-12T02:59:34.213101+0000 mgr.smithi153.asomxr (mgr.14195) 1820 : cluster 0 pgmap v1519: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:34.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:34 smithi153 bash[14340]: cluster 2024-03-12T02:59:34.213101+0000 mgr.smithi153.asomxr (mgr.14195) 1820 : cluster 0 pgmap v1519: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:37.525 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:37 smithi153 bash[14340]: cluster 2024-03-12T02:59:36.213711+0000 mgr.smithi153.asomxr (mgr.14195) 1821 : cluster 0 pgmap v1520: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:37.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:37 smithi163 bash[22347]: cluster 2024-03-12T02:59:36.213711+0000 mgr.smithi153.asomxr (mgr.14195) 1821 : cluster 0 pgmap v1520: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:38.291 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:59:39.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:39 smithi153 bash[14340]: cluster 2024-03-12T02:59:38.214296+0000 mgr.smithi153.asomxr (mgr.14195) 1822 : cluster 0 pgmap v1521: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:39.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:39 smithi163 bash[22347]: cluster 2024-03-12T02:59:38.214296+0000 mgr.smithi153.asomxr (mgr.14195) 1822 : cluster 0 pgmap v1521: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:41.555 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:59:41.555 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:59:41.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:41 smithi153 bash[14340]: cluster 2024-03-12T02:59:40.214947+0000 mgr.smithi153.asomxr (mgr.14195) 1823 : cluster 0 pgmap v1522: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:41.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:41 smithi163 bash[22347]: cluster 2024-03-12T02:59:40.214947+0000 mgr.smithi153.asomxr (mgr.14195) 1823 : cluster 0 pgmap v1522: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:42.355 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:59:43.356 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:59:43.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:43 smithi153 bash[14340]: audit 2024-03-12T02:59:41.537695+0000 mgr.smithi153.asomxr (mgr.14195) 1824 : audit 0 from='client.15514 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:43.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:43 smithi153 bash[14340]: cluster 2024-03-12T02:59:42.215564+0000 mgr.smithi153.asomxr (mgr.14195) 1825 : cluster 0 pgmap v1523: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:43.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:43 smithi163 bash[22347]: audit 2024-03-12T02:59:41.537695+0000 mgr.smithi153.asomxr (mgr.14195) 1824 : audit 0 from='client.15514 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:43.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:43 smithi163 bash[22347]: cluster 2024-03-12T02:59:42.215564+0000 mgr.smithi153.asomxr (mgr.14195) 1825 : cluster 0 pgmap v1523: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:44.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:44 smithi163 bash[22347]: cluster 2024-03-12T02:59:44.216116+0000 mgr.smithi153.asomxr (mgr.14195) 1826 : cluster 0 pgmap v1524: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:44.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:44 smithi153 bash[14340]: cluster 2024-03-12T02:59:44.216116+0000 mgr.smithi153.asomxr (mgr.14195) 1826 : cluster 0 pgmap v1524: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:45.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:45 smithi153 bash[14340]: audit 2024-03-12T02:59:45.454803+0000 mon.smithi153 (mon.0) 1244 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:59:45.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:45 smithi163 bash[22347]: audit 2024-03-12T02:59:45.454803+0000 mon.smithi153 (mon.0) 1244 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T02:59:46.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:46 smithi153 bash[14340]: cluster 2024-03-12T02:59:46.216776+0000 mgr.smithi153.asomxr (mgr.14195) 1827 : cluster 0 pgmap v1525: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:46.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:46 smithi163 bash[22347]: cluster 2024-03-12T02:59:46.216776+0000 mgr.smithi153.asomxr (mgr.14195) 1827 : cluster 0 pgmap v1525: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:48.106 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:59:49.405 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:49 smithi153 bash[14340]: cluster 2024-03-12T02:59:48.217212+0000 mgr.smithi153.asomxr (mgr.14195) 1828 : cluster 0 pgmap v1526: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:49.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:49 smithi163 bash[22347]: cluster 2024-03-12T02:59:48.217212+0000 mgr.smithi153.asomxr (mgr.14195) 1828 : cluster 0 pgmap v1526: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:51.343 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T02:59:51.343 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T02:59:51.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:51 smithi153 bash[14340]: cluster 2024-03-12T02:59:50.217778+0000 mgr.smithi153.asomxr (mgr.14195) 1829 : cluster 0 pgmap v1527: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:51.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:51 smithi163 bash[22347]: cluster 2024-03-12T02:59:50.217778+0000 mgr.smithi153.asomxr (mgr.14195) 1829 : cluster 0 pgmap v1527: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:52.221 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T02:59:52.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:52 smithi153 bash[14340]: audit 2024-03-12T02:59:51.329087+0000 mgr.smithi153.asomxr (mgr.14195) 1830 : audit 0 from='client.15518 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:52.654 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:52 smithi163 bash[22347]: audit 2024-03-12T02:59:51.329087+0000 mgr.smithi153.asomxr (mgr.14195) 1830 : audit 0 from='client.15518 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T02:59:53.222 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T02:59:53.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:53 smithi153 bash[14340]: cluster 2024-03-12T02:59:52.218420+0000 mgr.smithi153.asomxr (mgr.14195) 1831 : cluster 0 pgmap v1528: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:53.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:53 smithi163 bash[22347]: cluster 2024-03-12T02:59:52.218420+0000 mgr.smithi153.asomxr (mgr.14195) 1831 : cluster 0 pgmap v1528: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:54.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:54 smithi163 bash[22347]: cluster 2024-03-12T02:59:54.219060+0000 mgr.smithi153.asomxr (mgr.14195) 1832 : cluster 0 pgmap v1529: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:54.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:54 smithi153 bash[14340]: cluster 2024-03-12T02:59:54.219060+0000 mgr.smithi153.asomxr (mgr.14195) 1832 : cluster 0 pgmap v1529: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:57.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:57 smithi153 bash[14340]: cluster 2024-03-12T02:59:56.219757+0000 mgr.smithi153.asomxr (mgr.14195) 1833 : cluster 0 pgmap v1530: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:57.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:57 smithi163 bash[22347]: cluster 2024-03-12T02:59:56.219757+0000 mgr.smithi153.asomxr (mgr.14195) 1833 : cluster 0 pgmap v1530: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:57.980 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T02:59:59.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 02:59:59 smithi153 bash[14340]: cluster 2024-03-12T02:59:58.220480+0000 mgr.smithi153.asomxr (mgr.14195) 1834 : cluster 0 pgmap v1531: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T02:59:59.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 02:59:59 smithi163 bash[22347]: cluster 2024-03-12T02:59:58.220480+0000 mgr.smithi153.asomxr (mgr.14195) 1834 : cluster 0 pgmap v1531: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:00.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:00 smithi153 bash[14340]: cluster 2024-03-12T03:00:00.000196+0000 mon.smithi153 (mon.0) 1245 : cluster 3 overall HEALTH_WARN 2 failed cephadm daemon(s) 2024-03-12T03:00:00.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:00 smithi163 bash[22347]: cluster 2024-03-12T03:00:00.000196+0000 mon.smithi153 (mon.0) 1245 : cluster 3 overall HEALTH_WARN 2 failed cephadm daemon(s) 2024-03-12T03:00:01.242 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:00:01.243 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:00:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:01 smithi153 bash[14340]: cluster 2024-03-12T03:00:00.221237+0000 mgr.smithi153.asomxr (mgr.14195) 1835 : cluster 0 pgmap v1532: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:01 smithi153 bash[14340]: audit 2024-03-12T03:00:00.454902+0000 mon.smithi153 (mon.0) 1246 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:00:01.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:01 smithi163 bash[22347]: cluster 2024-03-12T03:00:00.221237+0000 mgr.smithi153.asomxr (mgr.14195) 1835 : cluster 0 pgmap v1532: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:01.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:01 smithi163 bash[22347]: audit 2024-03-12T03:00:00.454902+0000 mon.smithi153 (mon.0) 1246 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:00:02.059 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:00:02.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:02 smithi153 bash[14340]: audit 2024-03-12T03:00:01.228104+0000 mgr.smithi153.asomxr (mgr.14195) 1836 : audit 0 from='client.15522 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:02.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:02 smithi163 bash[22347]: audit 2024-03-12T03:00:01.228104+0000 mgr.smithi153.asomxr (mgr.14195) 1836 : audit 0 from='client.15522 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:03.061 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:00:03.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:03 smithi153 bash[14340]: cluster 2024-03-12T03:00:02.221910+0000 mgr.smithi153.asomxr (mgr.14195) 1837 : cluster 0 pgmap v1533: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:03.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:03 smithi153 bash[14340]: audit 2024-03-12T03:00:02.311049+0000 mon.smithi153 (mon.0) 1247 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:00:03.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:03 smithi153 bash[14340]: audit 2024-03-12T03:00:02.905080+0000 mon.smithi153 (mon.0) 1248 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:00:03.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:03 smithi153 bash[14340]: audit 2024-03-12T03:00:02.906868+0000 mon.smithi153 (mon.0) 1249 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:00:03.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:03 smithi153 bash[14340]: audit 2024-03-12T03:00:02.916293+0000 mon.smithi153 (mon.0) 1250 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:00:03.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:03 smithi153 bash[14340]: audit 2024-03-12T03:00:02.920749+0000 mon.smithi153 (mon.0) 1251 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:00:03.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:03 smithi163 bash[22347]: cluster 2024-03-12T03:00:02.221910+0000 mgr.smithi153.asomxr (mgr.14195) 1837 : cluster 0 pgmap v1533: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:03.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:03 smithi163 bash[22347]: audit 2024-03-12T03:00:02.311049+0000 mon.smithi153 (mon.0) 1247 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:00:03.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:03 smithi163 bash[22347]: audit 2024-03-12T03:00:02.905080+0000 mon.smithi153 (mon.0) 1248 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:00:03.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:03 smithi163 bash[22347]: audit 2024-03-12T03:00:02.906868+0000 mon.smithi153 (mon.0) 1249 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:00:03.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:03 smithi163 bash[22347]: audit 2024-03-12T03:00:02.916293+0000 mon.smithi153 (mon.0) 1250 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:00:03.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:03 smithi163 bash[22347]: audit 2024-03-12T03:00:02.920749+0000 mon.smithi153 (mon.0) 1251 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:00:04.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:04 smithi153 bash[14340]: cluster 2024-03-12T03:00:02.909922+0000 mgr.smithi153.asomxr (mgr.14195) 1838 : cluster 0 pgmap v1534: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:04.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:04 smithi153 bash[14340]: cluster 2024-03-12T03:00:02.910385+0000 mgr.smithi153.asomxr (mgr.14195) 1839 : cluster 0 pgmap v1535: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:04.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:04 smithi163 bash[22347]: cluster 2024-03-12T03:00:02.909922+0000 mgr.smithi153.asomxr (mgr.14195) 1838 : cluster 0 pgmap v1534: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:04.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:04 smithi163 bash[22347]: cluster 2024-03-12T03:00:02.910385+0000 mgr.smithi153.asomxr (mgr.14195) 1839 : cluster 0 pgmap v1535: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:06.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:06 smithi153 bash[14340]: cluster 2024-03-12T03:00:04.910923+0000 mgr.smithi153.asomxr (mgr.14195) 1840 : cluster 0 pgmap v1536: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:06.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:06 smithi163 bash[22347]: cluster 2024-03-12T03:00:04.910923+0000 mgr.smithi153.asomxr (mgr.14195) 1840 : cluster 0 pgmap v1536: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:07.815 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:00:08.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:08 smithi153 bash[14340]: cluster 2024-03-12T03:00:06.911563+0000 mgr.smithi153.asomxr (mgr.14195) 1841 : cluster 0 pgmap v1537: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:08.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:08 smithi163 bash[22347]: cluster 2024-03-12T03:00:06.911563+0000 mgr.smithi153.asomxr (mgr.14195) 1841 : cluster 0 pgmap v1537: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:09.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:09 smithi163 bash[22347]: cluster 2024-03-12T03:00:08.912244+0000 mgr.smithi153.asomxr (mgr.14195) 1842 : cluster 0 pgmap v1538: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:09.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:09 smithi153 bash[14340]: cluster 2024-03-12T03:00:08.912244+0000 mgr.smithi153.asomxr (mgr.14195) 1842 : cluster 0 pgmap v1538: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:11.282 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:00:11.283 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:00:12.083 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:00:12.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:11 smithi163 bash[22347]: cluster 2024-03-12T03:00:10.912860+0000 mgr.smithi153.asomxr (mgr.14195) 1843 : cluster 0 pgmap v1539: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:12.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:11 smithi163 bash[22347]: audit 2024-03-12T03:00:11.265444+0000 mgr.smithi153.asomxr (mgr.14195) 1844 : audit 0 from='client.15526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:12.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:11 smithi153 bash[14340]: cluster 2024-03-12T03:00:10.912860+0000 mgr.smithi153.asomxr (mgr.14195) 1843 : cluster 0 pgmap v1539: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:12.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:11 smithi153 bash[14340]: audit 2024-03-12T03:00:11.265444+0000 mgr.smithi153.asomxr (mgr.14195) 1844 : audit 0 from='client.15526 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:13.084 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:00:14.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:13 smithi163 bash[22347]: cluster 2024-03-12T03:00:12.913559+0000 mgr.smithi153.asomxr (mgr.14195) 1845 : cluster 0 pgmap v1540: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:14.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:13 smithi153 bash[14340]: cluster 2024-03-12T03:00:12.913559+0000 mgr.smithi153.asomxr (mgr.14195) 1845 : cluster 0 pgmap v1540: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:16.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:15 smithi163 bash[22347]: cluster 2024-03-12T03:00:14.914158+0000 mgr.smithi153.asomxr (mgr.14195) 1846 : cluster 0 pgmap v1541: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:16.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:15 smithi163 bash[22347]: audit 2024-03-12T03:00:15.455427+0000 mon.smithi153 (mon.0) 1252 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:00:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:15 smithi153 bash[14340]: cluster 2024-03-12T03:00:14.914158+0000 mgr.smithi153.asomxr (mgr.14195) 1846 : cluster 0 pgmap v1541: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:15 smithi153 bash[14340]: audit 2024-03-12T03:00:15.455427+0000 mon.smithi153 (mon.0) 1252 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:00:17.835 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:00:18.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:17 smithi153 bash[14340]: cluster 2024-03-12T03:00:16.914777+0000 mgr.smithi153.asomxr (mgr.14195) 1847 : cluster 0 pgmap v1542: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:18.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:17 smithi163 bash[22347]: cluster 2024-03-12T03:00:16.914777+0000 mgr.smithi153.asomxr (mgr.14195) 1847 : cluster 0 pgmap v1542: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:19.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:19 smithi153 bash[14340]: cluster 2024-03-12T03:00:18.915501+0000 mgr.smithi153.asomxr (mgr.14195) 1848 : cluster 0 pgmap v1543: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:19.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:19 smithi163 bash[22347]: cluster 2024-03-12T03:00:18.915501+0000 mgr.smithi153.asomxr (mgr.14195) 1848 : cluster 0 pgmap v1543: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:21.249 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:00:21.249 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:00:22.066 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:00:22.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:21 smithi163 bash[22347]: cluster 2024-03-12T03:00:20.916147+0000 mgr.smithi153.asomxr (mgr.14195) 1849 : cluster 0 pgmap v1544: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:22.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:21 smithi163 bash[22347]: audit 2024-03-12T03:00:21.234525+0000 mgr.smithi153.asomxr (mgr.14195) 1850 : audit 0 from='client.15530 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:22.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:21 smithi153 bash[14340]: cluster 2024-03-12T03:00:20.916147+0000 mgr.smithi153.asomxr (mgr.14195) 1849 : cluster 0 pgmap v1544: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:22.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:21 smithi153 bash[14340]: audit 2024-03-12T03:00:21.234525+0000 mgr.smithi153.asomxr (mgr.14195) 1850 : audit 0 from='client.15530 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:23.067 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:00:24.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:23 smithi163 bash[22347]: cluster 2024-03-12T03:00:22.916715+0000 mgr.smithi153.asomxr (mgr.14195) 1851 : cluster 0 pgmap v1545: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:24.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:23 smithi153 bash[14340]: cluster 2024-03-12T03:00:22.916715+0000 mgr.smithi153.asomxr (mgr.14195) 1851 : cluster 0 pgmap v1545: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:26.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:25 smithi153 bash[14340]: cluster 2024-03-12T03:00:24.917372+0000 mgr.smithi153.asomxr (mgr.14195) 1852 : cluster 0 pgmap v1546: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:26.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:25 smithi163 bash[22347]: cluster 2024-03-12T03:00:24.917372+0000 mgr.smithi153.asomxr (mgr.14195) 1852 : cluster 0 pgmap v1546: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:27.820 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:00:28.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:27 smithi153 bash[14340]: cluster 2024-03-12T03:00:26.917978+0000 mgr.smithi153.asomxr (mgr.14195) 1853 : cluster 0 pgmap v1547: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:28.422 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:27 smithi163 bash[22347]: cluster 2024-03-12T03:00:26.917978+0000 mgr.smithi153.asomxr (mgr.14195) 1853 : cluster 0 pgmap v1547: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:29.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:29 smithi163 bash[22347]: cluster 2024-03-12T03:00:28.918624+0000 mgr.smithi153.asomxr (mgr.14195) 1854 : cluster 0 pgmap v1548: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:29.765 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:29 smithi153 bash[14340]: cluster 2024-03-12T03:00:28.918624+0000 mgr.smithi153.asomxr (mgr.14195) 1854 : cluster 0 pgmap v1548: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:30.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:30 smithi163 bash[22347]: audit 2024-03-12T03:00:30.455657+0000 mon.smithi153 (mon.0) 1253 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:00:30.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:30 smithi153 bash[14340]: audit 2024-03-12T03:00:30.455657+0000 mon.smithi153 (mon.0) 1253 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:00:31.082 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:00:31.082 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:00:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:31 smithi153 bash[14340]: cluster 2024-03-12T03:00:30.919273+0000 mgr.smithi153.asomxr (mgr.14195) 1855 : cluster 0 pgmap v1549: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:31.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:31 smithi153 bash[14340]: audit 2024-03-12T03:00:31.068517+0000 mgr.smithi153.asomxr (mgr.14195) 1856 : audit 0 from='client.15534 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:31.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:31 smithi163 bash[22347]: cluster 2024-03-12T03:00:30.919273+0000 mgr.smithi153.asomxr (mgr.14195) 1855 : cluster 0 pgmap v1549: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:31.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:31 smithi163 bash[22347]: audit 2024-03-12T03:00:31.068517+0000 mgr.smithi153.asomxr (mgr.14195) 1856 : audit 0 from='client.15534 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:31.930 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:00:32.931 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:00:34.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:33 smithi163 bash[22347]: cluster 2024-03-12T03:00:32.919907+0000 mgr.smithi153.asomxr (mgr.14195) 1857 : cluster 0 pgmap v1550: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:34.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:33 smithi153 bash[14340]: cluster 2024-03-12T03:00:32.919907+0000 mgr.smithi153.asomxr (mgr.14195) 1857 : cluster 0 pgmap v1550: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:36.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:35 smithi153 bash[14340]: cluster 2024-03-12T03:00:34.920552+0000 mgr.smithi153.asomxr (mgr.14195) 1858 : cluster 0 pgmap v1551: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:36.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:35 smithi163 bash[22347]: cluster 2024-03-12T03:00:34.920552+0000 mgr.smithi153.asomxr (mgr.14195) 1858 : cluster 0 pgmap v1551: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:37.681 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:00:38.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:37 smithi153 bash[14340]: cluster 2024-03-12T03:00:36.921235+0000 mgr.smithi153.asomxr (mgr.14195) 1859 : cluster 0 pgmap v1552: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:38.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:37 smithi163 bash[22347]: cluster 2024-03-12T03:00:36.921235+0000 mgr.smithi153.asomxr (mgr.14195) 1859 : cluster 0 pgmap v1552: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:39.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:39 smithi163 bash[22347]: cluster 2024-03-12T03:00:38.921787+0000 mgr.smithi153.asomxr (mgr.14195) 1860 : cluster 0 pgmap v1553: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:39.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:39 smithi153 bash[14340]: cluster 2024-03-12T03:00:38.921787+0000 mgr.smithi153.asomxr (mgr.14195) 1860 : cluster 0 pgmap v1553: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:41.201 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:00:41.202 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:00:41.950 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:00:42.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:41 smithi163 bash[22347]: cluster 2024-03-12T03:00:40.922341+0000 mgr.smithi153.asomxr (mgr.14195) 1861 : cluster 0 pgmap v1554: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:42.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:41 smithi163 bash[22347]: audit 2024-03-12T03:00:41.191672+0000 mgr.smithi153.asomxr (mgr.14195) 1862 : audit 0 from='client.15538 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:42.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:41 smithi153 bash[14340]: cluster 2024-03-12T03:00:40.922341+0000 mgr.smithi153.asomxr (mgr.14195) 1861 : cluster 0 pgmap v1554: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:42.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:41 smithi153 bash[14340]: audit 2024-03-12T03:00:41.191672+0000 mgr.smithi153.asomxr (mgr.14195) 1862 : audit 0 from='client.15538 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:42.951 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:00:44.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:43 smithi163 bash[22347]: cluster 2024-03-12T03:00:42.922906+0000 mgr.smithi153.asomxr (mgr.14195) 1863 : cluster 0 pgmap v1555: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:44.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:43 smithi153 bash[14340]: cluster 2024-03-12T03:00:42.922906+0000 mgr.smithi153.asomxr (mgr.14195) 1863 : cluster 0 pgmap v1555: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:46.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:45 smithi153 bash[14340]: cluster 2024-03-12T03:00:44.923457+0000 mgr.smithi153.asomxr (mgr.14195) 1864 : cluster 0 pgmap v1556: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:46.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:45 smithi153 bash[14340]: audit 2024-03-12T03:00:45.456219+0000 mon.smithi153 (mon.0) 1254 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:00:46.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:45 smithi163 bash[22347]: cluster 2024-03-12T03:00:44.923457+0000 mgr.smithi153.asomxr (mgr.14195) 1864 : cluster 0 pgmap v1556: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:45 smithi163 bash[22347]: audit 2024-03-12T03:00:45.456219+0000 mon.smithi153 (mon.0) 1254 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:00:47.700 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:00:48.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:47 smithi153 bash[14340]: cluster 2024-03-12T03:00:46.924149+0000 mgr.smithi153.asomxr (mgr.14195) 1865 : cluster 0 pgmap v1557: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:48.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:47 smithi163 bash[22347]: cluster 2024-03-12T03:00:46.924149+0000 mgr.smithi153.asomxr (mgr.14195) 1865 : cluster 0 pgmap v1557: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:49.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:49 smithi163 bash[22347]: cluster 2024-03-12T03:00:48.924767+0000 mgr.smithi153.asomxr (mgr.14195) 1866 : cluster 0 pgmap v1558: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:49.765 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:49 smithi153 bash[14340]: cluster 2024-03-12T03:00:48.924767+0000 mgr.smithi153.asomxr (mgr.14195) 1866 : cluster 0 pgmap v1558: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:51.116 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:00:51.116 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:00:51.890 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:00:52.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:51 smithi153 bash[14340]: cluster 2024-03-12T03:00:50.925388+0000 mgr.smithi153.asomxr (mgr.14195) 1867 : cluster 0 pgmap v1559: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:52.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:51 smithi153 bash[14340]: audit 2024-03-12T03:00:51.101227+0000 mgr.smithi153.asomxr (mgr.14195) 1868 : audit 0 from='client.15542 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:51 smithi163 bash[22347]: cluster 2024-03-12T03:00:50.925388+0000 mgr.smithi153.asomxr (mgr.14195) 1867 : cluster 0 pgmap v1559: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:51 smithi163 bash[22347]: audit 2024-03-12T03:00:51.101227+0000 mgr.smithi153.asomxr (mgr.14195) 1868 : audit 0 from='client.15542 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:00:52.891 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:00:54.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:53 smithi163 bash[22347]: cluster 2024-03-12T03:00:52.925958+0000 mgr.smithi153.asomxr (mgr.14195) 1869 : cluster 0 pgmap v1560: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:54.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:53 smithi153 bash[14340]: cluster 2024-03-12T03:00:52.925958+0000 mgr.smithi153.asomxr (mgr.14195) 1869 : cluster 0 pgmap v1560: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:56.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:55 smithi153 bash[14340]: cluster 2024-03-12T03:00:54.926675+0000 mgr.smithi153.asomxr (mgr.14195) 1870 : cluster 0 pgmap v1561: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:56.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:55 smithi163 bash[22347]: cluster 2024-03-12T03:00:54.926675+0000 mgr.smithi153.asomxr (mgr.14195) 1870 : cluster 0 pgmap v1561: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:57.643 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:00:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:57 smithi153 bash[14340]: cluster 2024-03-12T03:00:56.927321+0000 mgr.smithi153.asomxr (mgr.14195) 1871 : cluster 0 pgmap v1562: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:58.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:57 smithi163 bash[22347]: cluster 2024-03-12T03:00:56.927321+0000 mgr.smithi153.asomxr (mgr.14195) 1871 : cluster 0 pgmap v1562: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:59.730 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:00:59 smithi153 bash[14340]: cluster 2024-03-12T03:00:58.927977+0000 mgr.smithi153.asomxr (mgr.14195) 1872 : cluster 0 pgmap v1563: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:00:59.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:00:59 smithi163 bash[22347]: cluster 2024-03-12T03:00:58.927977+0000 mgr.smithi153.asomxr (mgr.14195) 1872 : cluster 0 pgmap v1563: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:00.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:00 smithi153 bash[14340]: audit 2024-03-12T03:01:00.456563+0000 mon.smithi153 (mon.0) 1255 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:01:00.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:00 smithi163 bash[22347]: audit 2024-03-12T03:01:00.456563+0000 mon.smithi153 (mon.0) 1255 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:01:01.097 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:01:01.097 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:01:01.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:01 smithi153 bash[14340]: cluster 2024-03-12T03:01:00.928461+0000 mgr.smithi153.asomxr (mgr.14195) 1873 : cluster 0 pgmap v1564: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:01.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:01 smithi153 bash[14340]: audit 2024-03-12T03:01:01.087153+0000 mgr.smithi153.asomxr (mgr.14195) 1874 : audit 0 from='client.15546 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:01.953 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:01:01.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:01 smithi163 bash[22347]: cluster 2024-03-12T03:01:00.928461+0000 mgr.smithi153.asomxr (mgr.14195) 1873 : cluster 0 pgmap v1564: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:01.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:01 smithi163 bash[22347]: audit 2024-03-12T03:01:01.087153+0000 mgr.smithi153.asomxr (mgr.14195) 1874 : audit 0 from='client.15546 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:02.954 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:01:04.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:03 smithi153 bash[14340]: cluster 2024-03-12T03:01:02.929066+0000 mgr.smithi153.asomxr (mgr.14195) 1875 : cluster 0 pgmap v1565: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:04.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:03 smithi153 bash[14340]: audit 2024-03-12T03:01:03.013436+0000 mon.smithi153 (mon.0) 1256 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:01:04.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:03 smithi153 bash[14340]: audit 2024-03-12T03:01:03.606372+0000 mon.smithi153 (mon.0) 1257 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:01:04.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:03 smithi153 bash[14340]: audit 2024-03-12T03:01:03.608237+0000 mon.smithi153 (mon.0) 1258 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:01:04.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:03 smithi153 bash[14340]: audit 2024-03-12T03:01:03.619145+0000 mon.smithi153 (mon.0) 1259 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:01:04.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:03 smithi153 bash[14340]: audit 2024-03-12T03:01:03.623437+0000 mon.smithi153 (mon.0) 1260 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:01:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:03 smithi163 bash[22347]: cluster 2024-03-12T03:01:02.929066+0000 mgr.smithi153.asomxr (mgr.14195) 1875 : cluster 0 pgmap v1565: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:03 smithi163 bash[22347]: audit 2024-03-12T03:01:03.013436+0000 mon.smithi153 (mon.0) 1256 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:01:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:03 smithi163 bash[22347]: audit 2024-03-12T03:01:03.606372+0000 mon.smithi153 (mon.0) 1257 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:01:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:03 smithi163 bash[22347]: audit 2024-03-12T03:01:03.608237+0000 mon.smithi153 (mon.0) 1258 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:01:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:03 smithi163 bash[22347]: audit 2024-03-12T03:01:03.619145+0000 mon.smithi153 (mon.0) 1259 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:01:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:03 smithi163 bash[22347]: audit 2024-03-12T03:01:03.623437+0000 mon.smithi153 (mon.0) 1260 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:01:05.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:04 smithi153 bash[14340]: cluster 2024-03-12T03:01:03.611194+0000 mgr.smithi153.asomxr (mgr.14195) 1876 : cluster 0 pgmap v1566: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:05.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:04 smithi153 bash[14340]: cluster 2024-03-12T03:01:03.611618+0000 mgr.smithi153.asomxr (mgr.14195) 1877 : cluster 0 pgmap v1567: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:05.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:04 smithi163 bash[22347]: cluster 2024-03-12T03:01:03.611194+0000 mgr.smithi153.asomxr (mgr.14195) 1876 : cluster 0 pgmap v1566: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:05.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:04 smithi163 bash[22347]: cluster 2024-03-12T03:01:03.611618+0000 mgr.smithi153.asomxr (mgr.14195) 1877 : cluster 0 pgmap v1567: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:07.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:06 smithi153 bash[14340]: cluster 2024-03-12T03:01:05.612206+0000 mgr.smithi153.asomxr (mgr.14195) 1878 : cluster 0 pgmap v1568: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:07.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:06 smithi163 bash[22347]: cluster 2024-03-12T03:01:05.612206+0000 mgr.smithi153.asomxr (mgr.14195) 1878 : cluster 0 pgmap v1568: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:07.716 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:01:09.291 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:08 smithi153 bash[14340]: cluster 2024-03-12T03:01:07.612813+0000 mgr.smithi153.asomxr (mgr.14195) 1879 : cluster 0 pgmap v1569: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:09 smithi163 bash[22347]: cluster 2024-03-12T03:01:07.612813+0000 mgr.smithi153.asomxr (mgr.14195) 1879 : cluster 0 pgmap v1569: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:10.955 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:01:10.956 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:01:11.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:11 smithi153 bash[14340]: cluster 2024-03-12T03:01:09.613423+0000 mgr.smithi153.asomxr (mgr.14195) 1880 : cluster 0 pgmap v1570: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:11.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:11 smithi163 bash[22347]: cluster 2024-03-12T03:01:09.613423+0000 mgr.smithi153.asomxr (mgr.14195) 1880 : cluster 0 pgmap v1570: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:11.786 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:01:12.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:12 smithi153 bash[14340]: audit 2024-03-12T03:01:10.941036+0000 mgr.smithi153.asomxr (mgr.14195) 1881 : audit 0 from='client.15550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:12.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:12 smithi163 bash[22347]: audit 2024-03-12T03:01:10.941036+0000 mgr.smithi153.asomxr (mgr.14195) 1881 : audit 0 from='client.15550 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:12.788 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:01:13.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:13 smithi153 bash[14340]: cluster 2024-03-12T03:01:11.613997+0000 mgr.smithi153.asomxr (mgr.14195) 1882 : cluster 0 pgmap v1571: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:13.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:13 smithi163 bash[22347]: cluster 2024-03-12T03:01:11.613997+0000 mgr.smithi153.asomxr (mgr.14195) 1882 : cluster 0 pgmap v1571: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:14.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:14 smithi163 bash[22347]: cluster 2024-03-12T03:01:13.614610+0000 mgr.smithi153.asomxr (mgr.14195) 1883 : cluster 0 pgmap v1572: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:14.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:14 smithi153 bash[14340]: cluster 2024-03-12T03:01:13.614610+0000 mgr.smithi153.asomxr (mgr.14195) 1883 : cluster 0 pgmap v1572: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:15.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:15 smithi153 bash[14340]: audit 2024-03-12T03:01:15.457051+0000 mon.smithi153 (mon.0) 1261 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:01:15.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:15 smithi163 bash[22347]: audit 2024-03-12T03:01:15.457051+0000 mon.smithi153 (mon.0) 1261 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:01:16.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:16 smithi153 bash[14340]: cluster 2024-03-12T03:01:15.615257+0000 mgr.smithi153.asomxr (mgr.14195) 1884 : cluster 0 pgmap v1573: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:16.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:16 smithi163 bash[22347]: cluster 2024-03-12T03:01:15.615257+0000 mgr.smithi153.asomxr (mgr.14195) 1884 : cluster 0 pgmap v1573: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:17.536 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:01:18.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:18 smithi153 bash[14340]: cluster 2024-03-12T03:01:17.615987+0000 mgr.smithi153.asomxr (mgr.14195) 1885 : cluster 0 pgmap v1574: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:18.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:18 smithi163 bash[22347]: cluster 2024-03-12T03:01:17.615987+0000 mgr.smithi153.asomxr (mgr.14195) 1885 : cluster 0 pgmap v1574: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:20.773 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:01:20.773 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:01:20.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:20 smithi163 bash[22347]: cluster 2024-03-12T03:01:19.616760+0000 mgr.smithi153.asomxr (mgr.14195) 1886 : cluster 0 pgmap v1575: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:20.990 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:20 smithi153 bash[14340]: cluster 2024-03-12T03:01:19.616760+0000 mgr.smithi153.asomxr (mgr.14195) 1886 : cluster 0 pgmap v1575: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:21.574 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:01:21.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:21 smithi163 bash[22347]: audit 2024-03-12T03:01:20.758473+0000 mgr.smithi153.asomxr (mgr.14195) 1887 : audit 0 from='client.15554 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:22.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:21 smithi153 bash[14340]: audit 2024-03-12T03:01:20.758473+0000 mgr.smithi153.asomxr (mgr.14195) 1887 : audit 0 from='client.15554 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:22.575 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:01:22.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:22 smithi153 bash[14340]: cluster 2024-03-12T03:01:21.617441+0000 mgr.smithi153.asomxr (mgr.14195) 1888 : cluster 0 pgmap v1576: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:22.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:22 smithi163 bash[22347]: cluster 2024-03-12T03:01:21.617441+0000 mgr.smithi153.asomxr (mgr.14195) 1888 : cluster 0 pgmap v1576: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:24.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:24 smithi153 bash[14340]: cluster 2024-03-12T03:01:23.618102+0000 mgr.smithi153.asomxr (mgr.14195) 1889 : cluster 0 pgmap v1577: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:24.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:24 smithi163 bash[22347]: cluster 2024-03-12T03:01:23.618102+0000 mgr.smithi153.asomxr (mgr.14195) 1889 : cluster 0 pgmap v1577: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:26.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:26 smithi163 bash[22347]: cluster 2024-03-12T03:01:25.618809+0000 mgr.smithi153.asomxr (mgr.14195) 1890 : cluster 0 pgmap v1578: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:27.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:26 smithi153 bash[14340]: cluster 2024-03-12T03:01:25.618809+0000 mgr.smithi153.asomxr (mgr.14195) 1890 : cluster 0 pgmap v1578: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:27.332 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:01:28.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:28 smithi163 bash[22347]: cluster 2024-03-12T03:01:27.619463+0000 mgr.smithi153.asomxr (mgr.14195) 1891 : cluster 0 pgmap v1579: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:29.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:28 smithi153 bash[14340]: cluster 2024-03-12T03:01:27.619463+0000 mgr.smithi153.asomxr (mgr.14195) 1891 : cluster 0 pgmap v1579: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:30.656 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:01:30.656 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:01:30.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:30 smithi163 bash[22347]: cluster 2024-03-12T03:01:29.620077+0000 mgr.smithi153.asomxr (mgr.14195) 1892 : cluster 0 pgmap v1580: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:30.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:30 smithi163 bash[22347]: audit 2024-03-12T03:01:30.457324+0000 mon.smithi153 (mon.0) 1262 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:01:31.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:30 smithi153 bash[14340]: cluster 2024-03-12T03:01:29.620077+0000 mgr.smithi153.asomxr (mgr.14195) 1892 : cluster 0 pgmap v1580: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:31.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:30 smithi153 bash[14340]: audit 2024-03-12T03:01:30.457324+0000 mon.smithi153 (mon.0) 1262 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:01:31.527 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:01:31.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:31 smithi153 bash[14340]: audit 2024-03-12T03:01:30.641351+0000 mgr.smithi153.asomxr (mgr.14195) 1893 : audit 0 from='client.15558 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:31.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:31 smithi163 bash[22347]: audit 2024-03-12T03:01:30.641351+0000 mgr.smithi153.asomxr (mgr.14195) 1893 : audit 0 from='client.15558 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:32.528 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:01:32.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:32 smithi153 bash[14340]: cluster 2024-03-12T03:01:31.620716+0000 mgr.smithi153.asomxr (mgr.14195) 1894 : cluster 0 pgmap v1581: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:32.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:32 smithi163 bash[22347]: cluster 2024-03-12T03:01:31.620716+0000 mgr.smithi153.asomxr (mgr.14195) 1894 : cluster 0 pgmap v1581: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:34.745 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:34 smithi163 bash[22347]: cluster 2024-03-12T03:01:33.621370+0000 mgr.smithi153.asomxr (mgr.14195) 1895 : cluster 0 pgmap v1582: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:34.770 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:34 smithi153 bash[14340]: cluster 2024-03-12T03:01:33.621370+0000 mgr.smithi153.asomxr (mgr.14195) 1895 : cluster 0 pgmap v1582: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:36.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:36 smithi163 bash[22347]: cluster 2024-03-12T03:01:35.621968+0000 mgr.smithi153.asomxr (mgr.14195) 1896 : cluster 0 pgmap v1583: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:37.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:36 smithi153 bash[14340]: cluster 2024-03-12T03:01:35.621968+0000 mgr.smithi153.asomxr (mgr.14195) 1896 : cluster 0 pgmap v1583: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:37.282 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:01:38.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:38 smithi153 bash[14340]: cluster 2024-03-12T03:01:37.622526+0000 mgr.smithi153.asomxr (mgr.14195) 1897 : cluster 0 pgmap v1584: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:38.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:38 smithi163 bash[22347]: cluster 2024-03-12T03:01:37.622526+0000 mgr.smithi153.asomxr (mgr.14195) 1897 : cluster 0 pgmap v1584: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:40.468 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:01:40.468 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:01:40.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:40 smithi153 bash[14340]: cluster 2024-03-12T03:01:39.623160+0000 mgr.smithi153.asomxr (mgr.14195) 1898 : cluster 0 pgmap v1585: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:40.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:40 smithi163 bash[22347]: cluster 2024-03-12T03:01:39.623160+0000 mgr.smithi153.asomxr (mgr.14195) 1898 : cluster 0 pgmap v1585: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:41.280 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:01:41.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:41 smithi163 bash[22347]: audit 2024-03-12T03:01:40.453063+0000 mgr.smithi153.asomxr (mgr.14195) 1899 : audit 0 from='client.15562 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:42.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:41 smithi153 bash[14340]: audit 2024-03-12T03:01:40.453063+0000 mgr.smithi153.asomxr (mgr.14195) 1899 : audit 0 from='client.15562 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:42.280 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:01:42.961 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:42 smithi153 bash[14340]: cluster 2024-03-12T03:01:41.623740+0000 mgr.smithi153.asomxr (mgr.14195) 1900 : cluster 0 pgmap v1586: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:42.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:42 smithi163 bash[22347]: cluster 2024-03-12T03:01:41.623740+0000 mgr.smithi153.asomxr (mgr.14195) 1900 : cluster 0 pgmap v1586: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:44.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:44 smithi153 bash[14340]: cluster 2024-03-12T03:01:43.624379+0000 mgr.smithi153.asomxr (mgr.14195) 1901 : cluster 0 pgmap v1587: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:44.954 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:44 smithi163 bash[22347]: cluster 2024-03-12T03:01:43.624379+0000 mgr.smithi153.asomxr (mgr.14195) 1901 : cluster 0 pgmap v1587: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:45.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:45 smithi153 bash[14340]: audit 2024-03-12T03:01:45.457756+0000 mon.smithi153 (mon.0) 1263 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:01:45.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:45 smithi163 bash[22347]: audit 2024-03-12T03:01:45.457756+0000 mon.smithi153 (mon.0) 1263 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:01:46.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:46 smithi153 bash[14340]: cluster 2024-03-12T03:01:45.625057+0000 mgr.smithi153.asomxr (mgr.14195) 1902 : cluster 0 pgmap v1588: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:46.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:46 smithi163 bash[22347]: cluster 2024-03-12T03:01:45.625057+0000 mgr.smithi153.asomxr (mgr.14195) 1902 : cluster 0 pgmap v1588: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:47.037 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:01:48.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:48 smithi163 bash[22347]: cluster 2024-03-12T03:01:47.625667+0000 mgr.smithi153.asomxr (mgr.14195) 1903 : cluster 0 pgmap v1589: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:49.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:48 smithi153 bash[14340]: cluster 2024-03-12T03:01:47.625667+0000 mgr.smithi153.asomxr (mgr.14195) 1903 : cluster 0 pgmap v1589: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:50.220 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:01:50.220 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:01:50.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:50 smithi153 bash[14340]: cluster 2024-03-12T03:01:49.626094+0000 mgr.smithi153.asomxr (mgr.14195) 1904 : cluster 0 pgmap v1590: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:50.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:50 smithi153 bash[14340]: audit 2024-03-12T03:01:50.205738+0000 mgr.smithi153.asomxr (mgr.14195) 1905 : audit 0 from='client.15566 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:50.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:50 smithi163 bash[22347]: cluster 2024-03-12T03:01:49.626094+0000 mgr.smithi153.asomxr (mgr.14195) 1904 : cluster 0 pgmap v1590: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:50.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:50 smithi163 bash[22347]: audit 2024-03-12T03:01:50.205738+0000 mgr.smithi153.asomxr (mgr.14195) 1905 : audit 0 from='client.15566 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:01:51.244 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:01:52.245 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:01:52.959 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:52 smithi153 bash[14340]: cluster 2024-03-12T03:01:51.626669+0000 mgr.smithi153.asomxr (mgr.14195) 1906 : cluster 0 pgmap v1591: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:52.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:52 smithi163 bash[22347]: cluster 2024-03-12T03:01:51.626669+0000 mgr.smithi153.asomxr (mgr.14195) 1906 : cluster 0 pgmap v1591: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:54.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:54 smithi153 bash[14340]: cluster 2024-03-12T03:01:53.627327+0000 mgr.smithi153.asomxr (mgr.14195) 1907 : cluster 0 pgmap v1592: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:54.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:54 smithi163 bash[22347]: cluster 2024-03-12T03:01:53.627327+0000 mgr.smithi153.asomxr (mgr.14195) 1907 : cluster 0 pgmap v1592: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:56.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:56 smithi163 bash[22347]: cluster 2024-03-12T03:01:55.627978+0000 mgr.smithi153.asomxr (mgr.14195) 1908 : cluster 0 pgmap v1593: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:56.993 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:01:57.024 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:56 smithi153 bash[14340]: cluster 2024-03-12T03:01:55.627978+0000 mgr.smithi153.asomxr (mgr.14195) 1908 : cluster 0 pgmap v1593: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:58.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:01:58 smithi163 bash[22347]: cluster 2024-03-12T03:01:57.628648+0000 mgr.smithi153.asomxr (mgr.14195) 1909 : cluster 0 pgmap v1594: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:01:59.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:01:58 smithi153 bash[14340]: cluster 2024-03-12T03:01:57.628648+0000 mgr.smithi153.asomxr (mgr.14195) 1909 : cluster 0 pgmap v1594: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:00.108 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:02:00.108 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:02:00.884 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:02:00.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:00 smithi163 bash[22347]: cluster 2024-03-12T03:01:59.629376+0000 mgr.smithi153.asomxr (mgr.14195) 1910 : cluster 0 pgmap v1595: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:00.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:00 smithi163 bash[22347]: audit 2024-03-12T03:02:00.097940+0000 mgr.smithi153.asomxr (mgr.14195) 1911 : audit 0 from='client.15570 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:00.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:00 smithi163 bash[22347]: audit 2024-03-12T03:02:00.457546+0000 mon.smithi153 (mon.0) 1264 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:02:01.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:00 smithi153 bash[14340]: cluster 2024-03-12T03:01:59.629376+0000 mgr.smithi153.asomxr (mgr.14195) 1910 : cluster 0 pgmap v1595: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:01.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:00 smithi153 bash[14340]: audit 2024-03-12T03:02:00.097940+0000 mgr.smithi153.asomxr (mgr.14195) 1911 : audit 0 from='client.15570 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:01.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:00 smithi153 bash[14340]: audit 2024-03-12T03:02:00.457546+0000 mon.smithi153 (mon.0) 1264 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:02:01.885 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:02:02.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:02 smithi153 bash[14340]: cluster 2024-03-12T03:02:01.629988+0000 mgr.smithi153.asomxr (mgr.14195) 1912 : cluster 0 pgmap v1596: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:02 smithi163 bash[22347]: cluster 2024-03-12T03:02:01.629988+0000 mgr.smithi153.asomxr (mgr.14195) 1912 : cluster 0 pgmap v1596: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:04.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:04 smithi153 bash[14340]: cluster 2024-03-12T03:02:03.630576+0000 mgr.smithi153.asomxr (mgr.14195) 1913 : cluster 0 pgmap v1597: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:04.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:04 smithi153 bash[14340]: audit 2024-03-12T03:02:03.711023+0000 mon.smithi153 (mon.0) 1265 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:02:04.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:04 smithi153 bash[14340]: audit 2024-03-12T03:02:04.088868+0000 mon.smithi153 (mon.0) 1266 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:02:04.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:04 smithi153 bash[14340]: audit 2024-03-12T03:02:04.097450+0000 mon.smithi153 (mon.0) 1267 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:02:04.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:04 smithi163 bash[22347]: cluster 2024-03-12T03:02:03.630576+0000 mgr.smithi153.asomxr (mgr.14195) 1913 : cluster 0 pgmap v1597: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:04.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:04 smithi163 bash[22347]: audit 2024-03-12T03:02:03.711023+0000 mon.smithi153 (mon.0) 1265 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:02:04.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:04 smithi163 bash[22347]: audit 2024-03-12T03:02:04.088868+0000 mon.smithi153 (mon.0) 1266 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:02:04.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:04 smithi163 bash[22347]: audit 2024-03-12T03:02:04.097450+0000 mon.smithi153 (mon.0) 1267 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:02:06.638 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:02:06.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:06 smithi163 bash[22347]: cluster 2024-03-12T03:02:05.631236+0000 mgr.smithi153.asomxr (mgr.14195) 1914 : cluster 0 pgmap v1598: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:07.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:06 smithi153 bash[14340]: cluster 2024-03-12T03:02:05.631236+0000 mgr.smithi153.asomxr (mgr.14195) 1914 : cluster 0 pgmap v1598: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:08.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:08 smithi153 bash[14340]: cluster 2024-03-12T03:02:07.631793+0000 mgr.smithi153.asomxr (mgr.14195) 1915 : cluster 0 pgmap v1599: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:08.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:08 smithi163 bash[22347]: cluster 2024-03-12T03:02:07.631793+0000 mgr.smithi153.asomxr (mgr.14195) 1915 : cluster 0 pgmap v1599: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:09.810 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:02:09.810 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T02:51:50.721358Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:51:50.720742Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:51:50.720931Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T02:51:50.721558Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:51:50.722744Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T02:51:50.722937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:51:50.720523Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:51:50.720295Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:51:50.721125Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:51:50.721998Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T02:51:50.721763Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:02:10.549 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:02:10.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:10 smithi163 bash[22347]: cluster 2024-03-12T03:02:09.632294+0000 mgr.smithi153.asomxr (mgr.14195) 1916 : cluster 0 pgmap v1600: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:10.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:10 smithi163 bash[22347]: audit 2024-03-12T03:02:09.799353+0000 mgr.smithi153.asomxr (mgr.14195) 1917 : audit 0 from='client.15574 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:10.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:10 smithi163 bash[22347]: audit 2024-03-12T03:02:10.443314+0000 mon.smithi153 (mon.0) 1268 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:02:10.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:10 smithi163 bash[22347]: audit 2024-03-12T03:02:10.451857+0000 mon.smithi153 (mon.0) 1269 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:02:10.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:10 smithi153 bash[14340]: cluster 2024-03-12T03:02:09.632294+0000 mgr.smithi153.asomxr (mgr.14195) 1916 : cluster 0 pgmap v1600: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:10.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:10 smithi153 bash[14340]: audit 2024-03-12T03:02:09.799353+0000 mgr.smithi153.asomxr (mgr.14195) 1917 : audit 0 from='client.15574 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:10.831 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:10 smithi153 bash[14340]: audit 2024-03-12T03:02:10.443314+0000 mon.smithi153 (mon.0) 1268 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:02:10.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:10 smithi153 bash[14340]: audit 2024-03-12T03:02:10.451857+0000 mon.smithi153 (mon.0) 1269 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:02:11.550 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:02:11.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:11 smithi153 bash[14340]: audit 2024-03-12T03:02:10.974963+0000 mon.smithi153 (mon.0) 1270 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:02:11.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:11 smithi153 bash[14340]: audit 2024-03-12T03:02:10.976595+0000 mon.smithi153 (mon.0) 1271 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:02:11.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:11 smithi153 bash[14340]: cluster 2024-03-12T03:02:10.979310+0000 mgr.smithi153.asomxr (mgr.14195) 1918 : cluster 0 pgmap v1601: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:11.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:11 smithi153 bash[14340]: cluster 2024-03-12T03:02:10.979675+0000 mgr.smithi153.asomxr (mgr.14195) 1919 : cluster 0 pgmap v1602: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:11.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:11 smithi153 bash[14340]: audit 2024-03-12T03:02:10.986132+0000 mon.smithi153 (mon.0) 1272 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:02:11.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:11 smithi153 bash[14340]: audit 2024-03-12T03:02:10.990228+0000 mon.smithi153 (mon.0) 1273 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:02:11.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:11 smithi163 bash[22347]: audit 2024-03-12T03:02:10.974963+0000 mon.smithi153 (mon.0) 1270 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:02:11.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:11 smithi163 bash[22347]: audit 2024-03-12T03:02:10.976595+0000 mon.smithi153 (mon.0) 1271 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:02:11.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:11 smithi163 bash[22347]: cluster 2024-03-12T03:02:10.979310+0000 mgr.smithi153.asomxr (mgr.14195) 1918 : cluster 0 pgmap v1601: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:11.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:11 smithi163 bash[22347]: cluster 2024-03-12T03:02:10.979675+0000 mgr.smithi153.asomxr (mgr.14195) 1919 : cluster 0 pgmap v1602: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:11.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:11 smithi163 bash[22347]: audit 2024-03-12T03:02:10.986132+0000 mon.smithi153 (mon.0) 1272 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:02:11.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:11 smithi163 bash[22347]: audit 2024-03-12T03:02:10.990228+0000 mon.smithi153 (mon.0) 1273 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:02:14.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:14 smithi153 bash[14340]: cluster 2024-03-12T03:02:12.980261+0000 mgr.smithi153.asomxr (mgr.14195) 1920 : cluster 0 pgmap v1603: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:14.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:14 smithi163 bash[22347]: cluster 2024-03-12T03:02:12.980261+0000 mgr.smithi153.asomxr (mgr.14195) 1920 : cluster 0 pgmap v1603: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:16.305 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:02:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:16 smithi153 bash[14340]: cluster 2024-03-12T03:02:14.980999+0000 mgr.smithi153.asomxr (mgr.14195) 1921 : cluster 0 pgmap v1604: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:16.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:16 smithi153 bash[14340]: audit 2024-03-12T03:02:15.458592+0000 mon.smithi153 (mon.0) 1274 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:02:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:16 smithi163 bash[22347]: cluster 2024-03-12T03:02:14.980999+0000 mgr.smithi153.asomxr (mgr.14195) 1921 : cluster 0 pgmap v1604: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:16.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:16 smithi163 bash[22347]: audit 2024-03-12T03:02:15.458592+0000 mon.smithi153 (mon.0) 1274 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:02:18.310 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:18 smithi153 bash[14340]: cluster 2024-03-12T03:02:16.981612+0000 mgr.smithi153.asomxr (mgr.14195) 1922 : cluster 0 pgmap v1605: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:18.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:18 smithi163 bash[22347]: cluster 2024-03-12T03:02:16.981612+0000 mgr.smithi153.asomxr (mgr.14195) 1922 : cluster 0 pgmap v1605: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:19.643 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:02:19.644 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:02:19.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:19 smithi153 bash[14340]: cluster 2024-03-12T03:02:18.982233+0000 mgr.smithi153.asomxr (mgr.14195) 1923 : cluster 0 pgmap v1606: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:19.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:19 smithi163 bash[22347]: cluster 2024-03-12T03:02:18.982233+0000 mgr.smithi153.asomxr (mgr.14195) 1923 : cluster 0 pgmap v1606: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:20.400 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:02:20.667 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:20 smithi153 bash[14340]: audit 2024-03-12T03:02:19.632973+0000 mgr.smithi153.asomxr (mgr.14195) 1924 : audit 0 from='client.15578 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:20.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:20 smithi163 bash[22347]: audit 2024-03-12T03:02:19.632973+0000 mgr.smithi153.asomxr (mgr.14195) 1924 : audit 0 from='client.15578 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:21.402 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:02:21.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:21 smithi153 bash[14340]: cluster 2024-03-12T03:02:20.982876+0000 mgr.smithi153.asomxr (mgr.14195) 1925 : cluster 0 pgmap v1607: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:21.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:21 smithi163 bash[22347]: cluster 2024-03-12T03:02:20.982876+0000 mgr.smithi153.asomxr (mgr.14195) 1925 : cluster 0 pgmap v1607: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:24.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:24 smithi153 bash[14340]: cluster 2024-03-12T03:02:22.983517+0000 mgr.smithi153.asomxr (mgr.14195) 1926 : cluster 0 pgmap v1608: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:24.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:24 smithi163 bash[22347]: cluster 2024-03-12T03:02:22.983517+0000 mgr.smithi153.asomxr (mgr.14195) 1926 : cluster 0 pgmap v1608: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:26.158 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:02:26.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:26 smithi153 bash[14340]: cluster 2024-03-12T03:02:24.984146+0000 mgr.smithi153.asomxr (mgr.14195) 1927 : cluster 0 pgmap v1609: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:26.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:26 smithi163 bash[22347]: cluster 2024-03-12T03:02:24.984146+0000 mgr.smithi153.asomxr (mgr.14195) 1927 : cluster 0 pgmap v1609: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:28.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:28 smithi153 bash[14340]: cluster 2024-03-12T03:02:26.984779+0000 mgr.smithi153.asomxr (mgr.14195) 1928 : cluster 0 pgmap v1610: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:28 smithi163 bash[22347]: cluster 2024-03-12T03:02:26.984779+0000 mgr.smithi153.asomxr (mgr.14195) 1928 : cluster 0 pgmap v1610: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:29.470 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:02:29.471 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:02:29.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:29 smithi153 bash[14340]: cluster 2024-03-12T03:02:28.985437+0000 mgr.smithi153.asomxr (mgr.14195) 1929 : cluster 0 pgmap v1611: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:29.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:29 smithi163 bash[22347]: cluster 2024-03-12T03:02:28.985437+0000 mgr.smithi153.asomxr (mgr.14195) 1929 : cluster 0 pgmap v1611: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:30.233 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:02:30.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:30 smithi153 bash[14340]: audit 2024-03-12T03:02:29.453860+0000 mgr.smithi153.asomxr (mgr.14195) 1930 : audit 0 from='client.15582 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:30.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:30 smithi153 bash[14340]: audit 2024-03-12T03:02:30.458992+0000 mon.smithi153 (mon.0) 1275 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:02:30.884 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:30 smithi163 bash[22347]: audit 2024-03-12T03:02:29.453860+0000 mgr.smithi153.asomxr (mgr.14195) 1930 : audit 0 from='client.15582 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:30.884 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:30 smithi163 bash[22347]: audit 2024-03-12T03:02:30.458992+0000 mon.smithi153 (mon.0) 1275 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:02:31.234 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:02:31.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:31 smithi153 bash[14340]: cluster 2024-03-12T03:02:30.986047+0000 mgr.smithi153.asomxr (mgr.14195) 1931 : cluster 0 pgmap v1612: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:31.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:31 smithi163 bash[22347]: cluster 2024-03-12T03:02:30.986047+0000 mgr.smithi153.asomxr (mgr.14195) 1931 : cluster 0 pgmap v1612: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:34.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:34 smithi153 bash[14340]: cluster 2024-03-12T03:02:32.986659+0000 mgr.smithi153.asomxr (mgr.14195) 1932 : cluster 0 pgmap v1613: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:34.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:34 smithi163 bash[22347]: cluster 2024-03-12T03:02:32.986659+0000 mgr.smithi153.asomxr (mgr.14195) 1932 : cluster 0 pgmap v1613: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:35.980 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:02:36.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:36 smithi153 bash[14340]: cluster 2024-03-12T03:02:34.987288+0000 mgr.smithi153.asomxr (mgr.14195) 1933 : cluster 0 pgmap v1614: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:36.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:36 smithi163 bash[22347]: cluster 2024-03-12T03:02:34.987288+0000 mgr.smithi153.asomxr (mgr.14195) 1933 : cluster 0 pgmap v1614: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:38.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:38 smithi153 bash[14340]: cluster 2024-03-12T03:02:36.987920+0000 mgr.smithi153.asomxr (mgr.14195) 1934 : cluster 0 pgmap v1615: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:38.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:38 smithi163 bash[22347]: cluster 2024-03-12T03:02:36.987920+0000 mgr.smithi153.asomxr (mgr.14195) 1934 : cluster 0 pgmap v1615: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:39.428 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:02:39.428 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:02:39.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:39 smithi153 bash[14340]: cluster 2024-03-12T03:02:38.988656+0000 mgr.smithi153.asomxr (mgr.14195) 1935 : cluster 0 pgmap v1616: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:39.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:39 smithi153 bash[14340]: audit 2024-03-12T03:02:39.414412+0000 mgr.smithi153.asomxr (mgr.14195) 1936 : audit 0 from='client.15586 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:39.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:39 smithi163 bash[22347]: cluster 2024-03-12T03:02:38.988656+0000 mgr.smithi153.asomxr (mgr.14195) 1935 : cluster 0 pgmap v1616: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:39.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:39 smithi163 bash[22347]: audit 2024-03-12T03:02:39.414412+0000 mgr.smithi153.asomxr (mgr.14195) 1936 : audit 0 from='client.15586 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:40.232 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:02:41.233 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:02:42.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:42 smithi153 bash[14340]: cluster 2024-03-12T03:02:40.989332+0000 mgr.smithi153.asomxr (mgr.14195) 1937 : cluster 0 pgmap v1617: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:42.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:42 smithi163 bash[22347]: cluster 2024-03-12T03:02:40.989332+0000 mgr.smithi153.asomxr (mgr.14195) 1937 : cluster 0 pgmap v1617: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:44.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:44 smithi153 bash[14340]: cluster 2024-03-12T03:02:42.989900+0000 mgr.smithi153.asomxr (mgr.14195) 1938 : cluster 0 pgmap v1618: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:44.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:44 smithi163 bash[22347]: cluster 2024-03-12T03:02:42.989900+0000 mgr.smithi153.asomxr (mgr.14195) 1938 : cluster 0 pgmap v1618: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:45.987 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:02:46.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:46 smithi153 bash[14340]: cluster 2024-03-12T03:02:44.990538+0000 mgr.smithi153.asomxr (mgr.14195) 1939 : cluster 0 pgmap v1619: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:46.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:46 smithi153 bash[14340]: audit 2024-03-12T03:02:45.459424+0000 mon.smithi153 (mon.0) 1276 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:02:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:46 smithi163 bash[22347]: cluster 2024-03-12T03:02:44.990538+0000 mgr.smithi153.asomxr (mgr.14195) 1939 : cluster 0 pgmap v1619: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:46 smithi163 bash[22347]: audit 2024-03-12T03:02:45.459424+0000 mon.smithi153 (mon.0) 1276 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:02:48.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:48 smithi153 bash[14340]: cluster 2024-03-12T03:02:46.991145+0000 mgr.smithi153.asomxr (mgr.14195) 1940 : cluster 0 pgmap v1620: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:48.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:48 smithi163 bash[22347]: cluster 2024-03-12T03:02:46.991145+0000 mgr.smithi153.asomxr (mgr.14195) 1940 : cluster 0 pgmap v1620: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:49.174 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:02:49.174 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:02:49.807 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:49 smithi163 bash[22347]: cluster 2024-03-12T03:02:48.991773+0000 mgr.smithi153.asomxr (mgr.14195) 1941 : cluster 0 pgmap v1621: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:49.808 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:49 smithi163 bash[22347]: audit 2024-03-12T03:02:49.159823+0000 mgr.smithi153.asomxr (mgr.14195) 1942 : audit 0 from='client.15590 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:49.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:49 smithi153 bash[14340]: cluster 2024-03-12T03:02:48.991773+0000 mgr.smithi153.asomxr (mgr.14195) 1941 : cluster 0 pgmap v1621: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:49.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:49 smithi153 bash[14340]: audit 2024-03-12T03:02:49.159823+0000 mgr.smithi153.asomxr (mgr.14195) 1942 : audit 0 from='client.15590 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:49.946 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:02:50.947 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:02:52.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:52 smithi153 bash[14340]: cluster 2024-03-12T03:02:50.992390+0000 mgr.smithi153.asomxr (mgr.14195) 1943 : cluster 0 pgmap v1622: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:52.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:52 smithi163 bash[22347]: cluster 2024-03-12T03:02:50.992390+0000 mgr.smithi153.asomxr (mgr.14195) 1943 : cluster 0 pgmap v1622: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:54.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:54 smithi153 bash[14340]: cluster 2024-03-12T03:02:52.993026+0000 mgr.smithi153.asomxr (mgr.14195) 1944 : cluster 0 pgmap v1623: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:54 smithi163 bash[22347]: cluster 2024-03-12T03:02:52.993026+0000 mgr.smithi153.asomxr (mgr.14195) 1944 : cluster 0 pgmap v1623: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:55.700 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:02:56.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:56 smithi153 bash[14340]: cluster 2024-03-12T03:02:54.993669+0000 mgr.smithi153.asomxr (mgr.14195) 1945 : cluster 0 pgmap v1624: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:56.395 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:56 smithi163 bash[22347]: cluster 2024-03-12T03:02:54.993669+0000 mgr.smithi153.asomxr (mgr.14195) 1945 : cluster 0 pgmap v1624: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:58.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:58 smithi153 bash[14340]: cluster 2024-03-12T03:02:56.994231+0000 mgr.smithi153.asomxr (mgr.14195) 1946 : cluster 0 pgmap v1625: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:58.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:58 smithi163 bash[22347]: cluster 2024-03-12T03:02:56.994231+0000 mgr.smithi153.asomxr (mgr.14195) 1946 : cluster 0 pgmap v1625: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:59.003 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:02:59.004 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:02:59.758 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:02:59.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:59 smithi153 bash[14340]: audit 2024-03-12T03:02:58.989413+0000 mgr.smithi153.asomxr (mgr.14195) 1947 : audit 0 from='client.15594 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:59.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:02:59 smithi153 bash[14340]: cluster 2024-03-12T03:02:58.995002+0000 mgr.smithi153.asomxr (mgr.14195) 1948 : cluster 0 pgmap v1626: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:02:59.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:59 smithi163 bash[22347]: audit 2024-03-12T03:02:58.989413+0000 mgr.smithi153.asomxr (mgr.14195) 1947 : audit 0 from='client.15594 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:02:59.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:02:59 smithi163 bash[22347]: cluster 2024-03-12T03:02:58.995002+0000 mgr.smithi153.asomxr (mgr.14195) 1948 : cluster 0 pgmap v1626: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:00.759 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:03:00.771 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:00 smithi153 bash[14340]: audit 2024-03-12T03:03:00.459919+0000 mon.smithi153 (mon.0) 1277 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:03:00.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:00 smithi163 bash[22347]: audit 2024-03-12T03:03:00.459919+0000 mon.smithi153 (mon.0) 1277 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:03:01.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:01 smithi153 bash[14340]: cluster 2024-03-12T03:03:00.995764+0000 mgr.smithi153.asomxr (mgr.14195) 1949 : cluster 0 pgmap v1627: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:01.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:01 smithi163 bash[22347]: cluster 2024-03-12T03:03:00.995764+0000 mgr.smithi153.asomxr (mgr.14195) 1949 : cluster 0 pgmap v1627: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:04.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:04 smithi153 bash[14340]: cluster 2024-03-12T03:03:02.996358+0000 mgr.smithi153.asomxr (mgr.14195) 1950 : cluster 0 pgmap v1628: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:04 smithi163 bash[22347]: cluster 2024-03-12T03:03:02.996358+0000 mgr.smithi153.asomxr (mgr.14195) 1950 : cluster 0 pgmap v1628: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:05.512 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:03:06.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:06 smithi153 bash[14340]: cluster 2024-03-12T03:03:04.996855+0000 mgr.smithi153.asomxr (mgr.14195) 1951 : cluster 0 pgmap v1629: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:06.482 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:06 smithi163 bash[22347]: cluster 2024-03-12T03:03:04.996855+0000 mgr.smithi153.asomxr (mgr.14195) 1951 : cluster 0 pgmap v1629: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:08.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:08 smithi153 bash[14340]: cluster 2024-03-12T03:03:06.997723+0000 mgr.smithi153.asomxr (mgr.14195) 1952 : cluster 0 pgmap v1630: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:08.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:08 smithi163 bash[22347]: cluster 2024-03-12T03:03:06.997723+0000 mgr.smithi153.asomxr (mgr.14195) 1952 : cluster 0 pgmap v1630: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:08.864 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:03:08.864 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:03:09.673 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:03:09.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:09 smithi153 bash[14340]: audit 2024-03-12T03:03:08.849648+0000 mgr.smithi153.asomxr (mgr.14195) 1953 : audit 0 from='client.15598 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:03:09.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:09 smithi153 bash[14340]: cluster 2024-03-12T03:03:08.998364+0000 mgr.smithi153.asomxr (mgr.14195) 1954 : cluster 0 pgmap v1631: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:09.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:09 smithi163 bash[22347]: audit 2024-03-12T03:03:08.849648+0000 mgr.smithi153.asomxr (mgr.14195) 1953 : audit 0 from='client.15598 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:03:09.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:09 smithi163 bash[22347]: cluster 2024-03-12T03:03:08.998364+0000 mgr.smithi153.asomxr (mgr.14195) 1954 : cluster 0 pgmap v1631: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:10.674 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:03:12.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:12 smithi153 bash[14340]: cluster 2024-03-12T03:03:10.999028+0000 mgr.smithi153.asomxr (mgr.14195) 1955 : cluster 0 pgmap v1632: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:12.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:12 smithi153 bash[14340]: audit 2024-03-12T03:03:11.082055+0000 mon.smithi153 (mon.0) 1278 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:03:12.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:12 smithi153 bash[14340]: audit 2024-03-12T03:03:11.658334+0000 mon.smithi153 (mon.0) 1279 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:03:12.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:12 smithi153 bash[14340]: audit 2024-03-12T03:03:11.659650+0000 mon.smithi153 (mon.0) 1280 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:03:12.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:12 smithi153 bash[14340]: audit 2024-03-12T03:03:11.666704+0000 mon.smithi153 (mon.0) 1281 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:03:12.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:12 smithi153 bash[14340]: audit 2024-03-12T03:03:11.672299+0000 mon.smithi153 (mon.0) 1282 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:03:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:12 smithi163 bash[22347]: cluster 2024-03-12T03:03:10.999028+0000 mgr.smithi153.asomxr (mgr.14195) 1955 : cluster 0 pgmap v1632: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:12 smithi163 bash[22347]: audit 2024-03-12T03:03:11.082055+0000 mon.smithi153 (mon.0) 1278 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:03:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:12 smithi163 bash[22347]: audit 2024-03-12T03:03:11.658334+0000 mon.smithi153 (mon.0) 1279 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:03:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:12 smithi163 bash[22347]: audit 2024-03-12T03:03:11.659650+0000 mon.smithi153 (mon.0) 1280 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:03:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:12 smithi163 bash[22347]: audit 2024-03-12T03:03:11.666704+0000 mon.smithi153 (mon.0) 1281 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:03:12.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:12 smithi163 bash[22347]: audit 2024-03-12T03:03:11.672299+0000 mon.smithi153 (mon.0) 1282 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:03:13.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:13 smithi153 bash[14340]: cluster 2024-03-12T03:03:11.661639+0000 mgr.smithi153.asomxr (mgr.14195) 1956 : cluster 0 pgmap v1633: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:13.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:13 smithi153 bash[14340]: cluster 2024-03-12T03:03:11.661959+0000 mgr.smithi153.asomxr (mgr.14195) 1957 : cluster 0 pgmap v1634: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:13.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:13 smithi163 bash[22347]: cluster 2024-03-12T03:03:11.661639+0000 mgr.smithi153.asomxr (mgr.14195) 1956 : cluster 0 pgmap v1633: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:13.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:13 smithi163 bash[22347]: cluster 2024-03-12T03:03:11.661959+0000 mgr.smithi153.asomxr (mgr.14195) 1957 : cluster 0 pgmap v1634: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:14.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:14 smithi153 bash[14340]: cluster 2024-03-12T03:03:13.662575+0000 mgr.smithi153.asomxr (mgr.14195) 1958 : cluster 0 pgmap v1635: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:14.982 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:14 smithi163 bash[22347]: cluster 2024-03-12T03:03:13.662575+0000 mgr.smithi153.asomxr (mgr.14195) 1958 : cluster 0 pgmap v1635: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:15.431 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:03:15.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:15 smithi153 bash[14340]: audit 2024-03-12T03:03:15.460025+0000 mon.smithi153 (mon.0) 1283 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:03:15.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:15 smithi163 bash[22347]: audit 2024-03-12T03:03:15.460025+0000 mon.smithi153 (mon.0) 1283 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:03:16.799 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:16 smithi163 bash[22347]: cluster 2024-03-12T03:03:15.663210+0000 mgr.smithi153.asomxr (mgr.14195) 1959 : cluster 0 pgmap v1636: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:16.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:16 smithi153 bash[14340]: cluster 2024-03-12T03:03:15.663210+0000 mgr.smithi153.asomxr (mgr.14195) 1959 : cluster 0 pgmap v1636: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:18.854 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:03:18.854 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:03:18.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:18 smithi163 bash[22347]: cluster 2024-03-12T03:03:17.663837+0000 mgr.smithi153.asomxr (mgr.14195) 1960 : cluster 0 pgmap v1637: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:19.036 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:18 smithi153 bash[14340]: cluster 2024-03-12T03:03:17.663837+0000 mgr.smithi153.asomxr (mgr.14195) 1960 : cluster 0 pgmap v1637: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:19.523 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:03:19.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:19 smithi153 bash[14340]: audit 2024-03-12T03:03:18.844384+0000 mgr.smithi153.asomxr (mgr.14195) 1961 : audit 0 from='client.15602 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:03:19.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:19 smithi163 bash[22347]: audit 2024-03-12T03:03:18.844384+0000 mgr.smithi153.asomxr (mgr.14195) 1961 : audit 0 from='client.15602 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:03:20.524 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:03:20.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:20 smithi153 bash[14340]: cluster 2024-03-12T03:03:19.664490+0000 mgr.smithi153.asomxr (mgr.14195) 1962 : cluster 0 pgmap v1638: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:21.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:20 smithi163 bash[22347]: cluster 2024-03-12T03:03:19.664490+0000 mgr.smithi153.asomxr (mgr.14195) 1962 : cluster 0 pgmap v1638: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:23.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:22 smithi153 bash[14340]: cluster 2024-03-12T03:03:21.665102+0000 mgr.smithi153.asomxr (mgr.14195) 1963 : cluster 0 pgmap v1639: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:23.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:22 smithi163 bash[22347]: cluster 2024-03-12T03:03:21.665102+0000 mgr.smithi153.asomxr (mgr.14195) 1963 : cluster 0 pgmap v1639: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:24.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:24 smithi153 bash[14340]: cluster 2024-03-12T03:03:23.665704+0000 mgr.smithi153.asomxr (mgr.14195) 1964 : cluster 0 pgmap v1640: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:24.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:24 smithi163 bash[22347]: cluster 2024-03-12T03:03:23.665704+0000 mgr.smithi153.asomxr (mgr.14195) 1964 : cluster 0 pgmap v1640: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:25.277 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:03:26.912 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:26 smithi153 bash[14340]: cluster 2024-03-12T03:03:25.666106+0000 mgr.smithi153.asomxr (mgr.14195) 1965 : cluster 0 pgmap v1641: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:26.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:26 smithi163 bash[22347]: cluster 2024-03-12T03:03:25.666106+0000 mgr.smithi153.asomxr (mgr.14195) 1965 : cluster 0 pgmap v1641: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:28.663 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:03:28.663 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:03:28.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:28 smithi163 bash[22347]: cluster 2024-03-12T03:03:27.666666+0000 mgr.smithi153.asomxr (mgr.14195) 1966 : cluster 0 pgmap v1642: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:29.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:28 smithi153 bash[14340]: cluster 2024-03-12T03:03:27.666666+0000 mgr.smithi153.asomxr (mgr.14195) 1966 : cluster 0 pgmap v1642: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:29.491 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:03:29.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:29 smithi153 bash[14340]: audit 2024-03-12T03:03:28.648523+0000 mgr.smithi153.asomxr (mgr.14195) 1967 : audit 0 from='client.15606 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:03:30.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:29 smithi163 bash[22347]: audit 2024-03-12T03:03:28.648523+0000 mgr.smithi153.asomxr (mgr.14195) 1967 : audit 0 from='client.15606 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:03:30.491 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:03:30.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:30 smithi153 bash[14340]: cluster 2024-03-12T03:03:29.667343+0000 mgr.smithi153.asomxr (mgr.14195) 1968 : cluster 0 pgmap v1643: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:30.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:30 smithi153 bash[14340]: audit 2024-03-12T03:03:30.460560+0000 mon.smithi153 (mon.0) 1284 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:03:31.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:30 smithi163 bash[22347]: cluster 2024-03-12T03:03:29.667343+0000 mgr.smithi153.asomxr (mgr.14195) 1968 : cluster 0 pgmap v1643: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:31.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:30 smithi163 bash[22347]: audit 2024-03-12T03:03:30.460560+0000 mon.smithi153 (mon.0) 1284 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:03:33.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:32 smithi153 bash[14340]: cluster 2024-03-12T03:03:31.667842+0000 mgr.smithi153.asomxr (mgr.14195) 1969 : cluster 0 pgmap v1644: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:33.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:32 smithi163 bash[22347]: cluster 2024-03-12T03:03:31.667842+0000 mgr.smithi153.asomxr (mgr.14195) 1969 : cluster 0 pgmap v1644: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:34.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:34 smithi153 bash[14340]: cluster 2024-03-12T03:03:33.668463+0000 mgr.smithi153.asomxr (mgr.14195) 1970 : cluster 0 pgmap v1645: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:34.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:34 smithi163 bash[22347]: cluster 2024-03-12T03:03:33.668463+0000 mgr.smithi153.asomxr (mgr.14195) 1970 : cluster 0 pgmap v1645: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:35.246 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:03:36.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:36 smithi163 bash[22347]: cluster 2024-03-12T03:03:35.669075+0000 mgr.smithi153.asomxr (mgr.14195) 1971 : cluster 0 pgmap v1646: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:37.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:36 smithi153 bash[14340]: cluster 2024-03-12T03:03:35.669075+0000 mgr.smithi153.asomxr (mgr.14195) 1971 : cluster 0 pgmap v1646: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:38.618 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:03:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:03:39.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:38 smithi153 bash[14340]: cluster 2024-03-12T03:03:37.669693+0000 mgr.smithi153.asomxr (mgr.14195) 1972 : cluster 0 pgmap v1647: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:39.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:38 smithi163 bash[22347]: cluster 2024-03-12T03:03:37.669693+0000 mgr.smithi153.asomxr (mgr.14195) 1972 : cluster 0 pgmap v1647: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:39.406 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:03:40.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:39 smithi153 bash[14340]: audit 2024-03-12T03:03:38.604955+0000 mgr.smithi153.asomxr (mgr.14195) 1973 : audit 0 from='client.15610 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:03:40.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:39 smithi163 bash[22347]: audit 2024-03-12T03:03:38.604955+0000 mgr.smithi153.asomxr (mgr.14195) 1973 : audit 0 from='client.15610 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:03:40.407 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:03:41.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:40 smithi153 bash[14340]: cluster 2024-03-12T03:03:39.670245+0000 mgr.smithi153.asomxr (mgr.14195) 1974 : cluster 0 pgmap v1648: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:41.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:40 smithi163 bash[22347]: cluster 2024-03-12T03:03:39.670245+0000 mgr.smithi153.asomxr (mgr.14195) 1974 : cluster 0 pgmap v1648: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:43.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:42 smithi153 bash[14340]: cluster 2024-03-12T03:03:41.670832+0000 mgr.smithi153.asomxr (mgr.14195) 1975 : cluster 0 pgmap v1649: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:43.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:42 smithi163 bash[22347]: cluster 2024-03-12T03:03:41.670832+0000 mgr.smithi153.asomxr (mgr.14195) 1975 : cluster 0 pgmap v1649: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:44.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:44 smithi153 bash[14340]: cluster 2024-03-12T03:03:43.671399+0000 mgr.smithi153.asomxr (mgr.14195) 1976 : cluster 0 pgmap v1650: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:44.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:44 smithi163 bash[22347]: cluster 2024-03-12T03:03:43.671399+0000 mgr.smithi153.asomxr (mgr.14195) 1976 : cluster 0 pgmap v1650: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:45.165 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:03:45.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:45 smithi153 bash[14340]: audit 2024-03-12T03:03:45.461011+0000 mon.smithi153 (mon.0) 1285 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:03:45.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:45 smithi163 bash[22347]: audit 2024-03-12T03:03:45.461011+0000 mon.smithi153 (mon.0) 1285 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:03:46.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:46 smithi153 bash[14340]: cluster 2024-03-12T03:03:45.671821+0000 mgr.smithi153.asomxr (mgr.14195) 1977 : cluster 0 pgmap v1651: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:46.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:46 smithi163 bash[22347]: cluster 2024-03-12T03:03:45.671821+0000 mgr.smithi153.asomxr (mgr.14195) 1977 : cluster 0 pgmap v1651: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:48.535 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:03:48.536 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:03:48.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:48 smithi153 bash[14340]: cluster 2024-03-12T03:03:47.672490+0000 mgr.smithi153.asomxr (mgr.14195) 1978 : cluster 0 pgmap v1652: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:48.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:48 smithi163 bash[22347]: cluster 2024-03-12T03:03:47.672490+0000 mgr.smithi153.asomxr (mgr.14195) 1978 : cluster 0 pgmap v1652: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:49.356 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:03:50.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:49 smithi153 bash[14340]: audit 2024-03-12T03:03:48.520378+0000 mgr.smithi153.asomxr (mgr.14195) 1979 : audit 0 from='client.15614 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:03:50.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:49 smithi163 bash[22347]: audit 2024-03-12T03:03:48.520378+0000 mgr.smithi153.asomxr (mgr.14195) 1979 : audit 0 from='client.15614 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:03:50.357 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:03:51.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:50 smithi153 bash[14340]: cluster 2024-03-12T03:03:49.673282+0000 mgr.smithi153.asomxr (mgr.14195) 1980 : cluster 0 pgmap v1653: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:51.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:50 smithi163 bash[22347]: cluster 2024-03-12T03:03:49.673282+0000 mgr.smithi153.asomxr (mgr.14195) 1980 : cluster 0 pgmap v1653: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:53.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:52 smithi153 bash[14340]: cluster 2024-03-12T03:03:51.674053+0000 mgr.smithi153.asomxr (mgr.14195) 1981 : cluster 0 pgmap v1654: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:53.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:52 smithi163 bash[22347]: cluster 2024-03-12T03:03:51.674053+0000 mgr.smithi153.asomxr (mgr.14195) 1981 : cluster 0 pgmap v1654: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:54.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:54 smithi153 bash[14340]: cluster 2024-03-12T03:03:53.674614+0000 mgr.smithi153.asomxr (mgr.14195) 1982 : cluster 0 pgmap v1655: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:54.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:54 smithi163 bash[22347]: cluster 2024-03-12T03:03:53.674614+0000 mgr.smithi153.asomxr (mgr.14195) 1982 : cluster 0 pgmap v1655: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:55.114 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:03:57.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:56 smithi153 bash[14340]: cluster 2024-03-12T03:03:55.675284+0000 mgr.smithi153.asomxr (mgr.14195) 1983 : cluster 0 pgmap v1656: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:57.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:56 smithi163 bash[22347]: cluster 2024-03-12T03:03:55.675284+0000 mgr.smithi153.asomxr (mgr.14195) 1983 : cluster 0 pgmap v1656: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:58.550 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:03:58.550 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:03:59.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:58 smithi153 bash[14340]: cluster 2024-03-12T03:03:57.675840+0000 mgr.smithi153.asomxr (mgr.14195) 1984 : cluster 0 pgmap v1657: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:59.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:58 smithi163 bash[22347]: cluster 2024-03-12T03:03:57.675840+0000 mgr.smithi153.asomxr (mgr.14195) 1984 : cluster 0 pgmap v1657: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:03:59.336 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:04:00.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:03:59 smithi153 bash[14340]: audit 2024-03-12T03:03:58.536390+0000 mgr.smithi153.asomxr (mgr.14195) 1985 : audit 0 from='client.15618 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:00.234 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:03:59 smithi163 bash[22347]: audit 2024-03-12T03:03:58.536390+0000 mgr.smithi153.asomxr (mgr.14195) 1985 : audit 0 from='client.15618 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:00.337 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:04:01.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:00 smithi153 bash[14340]: cluster 2024-03-12T03:03:59.676475+0000 mgr.smithi153.asomxr (mgr.14195) 1986 : cluster 0 pgmap v1658: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:01.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:00 smithi153 bash[14340]: audit 2024-03-12T03:04:00.461293+0000 mon.smithi153 (mon.0) 1286 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:04:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:00 smithi163 bash[22347]: cluster 2024-03-12T03:03:59.676475+0000 mgr.smithi153.asomxr (mgr.14195) 1986 : cluster 0 pgmap v1658: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:01.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:00 smithi163 bash[22347]: audit 2024-03-12T03:04:00.461293+0000 mon.smithi153 (mon.0) 1286 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:04:02.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:02 smithi153 bash[14340]: cluster 2024-03-12T03:04:01.677115+0000 mgr.smithi153.asomxr (mgr.14195) 1987 : cluster 0 pgmap v1659: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:02.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:02 smithi163 bash[22347]: cluster 2024-03-12T03:04:01.677115+0000 mgr.smithi153.asomxr (mgr.14195) 1987 : cluster 0 pgmap v1659: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:04.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:04 smithi153 bash[14340]: cluster 2024-03-12T03:04:03.677745+0000 mgr.smithi153.asomxr (mgr.14195) 1988 : cluster 0 pgmap v1660: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:04.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:04 smithi163 bash[22347]: cluster 2024-03-12T03:04:03.677745+0000 mgr.smithi153.asomxr (mgr.14195) 1988 : cluster 0 pgmap v1660: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:05.081 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:04:07.021 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:06 smithi153 bash[14340]: cluster 2024-03-12T03:04:05.678412+0000 mgr.smithi153.asomxr (mgr.14195) 1989 : cluster 0 pgmap v1661: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:07.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:06 smithi163 bash[22347]: cluster 2024-03-12T03:04:05.678412+0000 mgr.smithi153.asomxr (mgr.14195) 1989 : cluster 0 pgmap v1661: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:08.454 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:04:08.454 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T02:53:58.253429Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T02:53:58.253713Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T02:53:58.255795Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T02:53:58.255555Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T02:53:58.256034Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T02:53:58.254208Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T02:53:58.254447Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T02:53:58.253965Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T02:53:58.254686Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:04:09.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:08 smithi153 bash[14340]: cluster 2024-03-12T03:04:07.679041+0000 mgr.smithi153.asomxr (mgr.14195) 1990 : cluster 0 pgmap v1662: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:09.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:08 smithi153 bash[14340]: audit 2024-03-12T03:04:08.440448+0000 mgr.smithi153.asomxr (mgr.14195) 1991 : audit 0 from='client.15622 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:09.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:08 smithi163 bash[22347]: cluster 2024-03-12T03:04:07.679041+0000 mgr.smithi153.asomxr (mgr.14195) 1990 : cluster 0 pgmap v1662: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:09.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:08 smithi163 bash[22347]: audit 2024-03-12T03:04:08.440448+0000 mgr.smithi153.asomxr (mgr.14195) 1991 : audit 0 from='client.15622 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:09.250 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:04:10.250 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:04:11.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:10 smithi153 bash[14340]: cluster 2024-03-12T03:04:09.679688+0000 mgr.smithi153.asomxr (mgr.14195) 1992 : cluster 0 pgmap v1663: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:11.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:10 smithi163 bash[22347]: cluster 2024-03-12T03:04:09.679688+0000 mgr.smithi153.asomxr (mgr.14195) 1992 : cluster 0 pgmap v1663: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:13.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:12 smithi153 bash[14340]: cluster 2024-03-12T03:04:11.680276+0000 mgr.smithi153.asomxr (mgr.14195) 1993 : cluster 0 pgmap v1664: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:13.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:12 smithi153 bash[14340]: audit 2024-03-12T03:04:11.763767+0000 mon.smithi153 (mon.0) 1287 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:04:13.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:12 smithi153 bash[14340]: audit 2024-03-12T03:04:12.140981+0000 mon.smithi153 (mon.0) 1288 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:04:13.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:12 smithi153 bash[14340]: audit 2024-03-12T03:04:12.148953+0000 mon.smithi153 (mon.0) 1289 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:04:13.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:12 smithi163 bash[22347]: cluster 2024-03-12T03:04:11.680276+0000 mgr.smithi153.asomxr (mgr.14195) 1993 : cluster 0 pgmap v1664: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:13.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:12 smithi163 bash[22347]: audit 2024-03-12T03:04:11.763767+0000 mon.smithi153 (mon.0) 1287 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:04:13.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:12 smithi163 bash[22347]: audit 2024-03-12T03:04:12.140981+0000 mon.smithi153 (mon.0) 1288 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:04:13.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:12 smithi163 bash[22347]: audit 2024-03-12T03:04:12.148953+0000 mon.smithi153 (mon.0) 1289 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:04:14.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:14 smithi153 bash[14340]: cluster 2024-03-12T03:04:13.680942+0000 mgr.smithi153.asomxr (mgr.14195) 1994 : cluster 0 pgmap v1665: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:14.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:14 smithi163 bash[22347]: cluster 2024-03-12T03:04:13.680942+0000 mgr.smithi153.asomxr (mgr.14195) 1994 : cluster 0 pgmap v1665: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:04:15.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:15 smithi153 bash[14340]: audit 2024-03-12T03:04:15.462032+0000 mon.smithi153 (mon.0) 1290 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:04:15.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:15 smithi163 bash[22347]: audit 2024-03-12T03:04:15.462032+0000 mon.smithi153 (mon.0) 1290 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:04:16.826 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:16 smithi153 bash[14340]: cluster 2024-03-12T03:04:15.681619+0000 mgr.smithi153.asomxr (mgr.14195) 1995 : cluster 0 pgmap v1666: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:16.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:16 smithi163 bash[22347]: cluster 2024-03-12T03:04:15.681619+0000 mgr.smithi153.asomxr (mgr.14195) 1995 : cluster 0 pgmap v1666: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:18.194 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:04:18.195 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:04:18.979 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:04:19.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:19 smithi153 bash[14340]: cluster 2024-03-12T03:04:17.682168+0000 mgr.smithi153.asomxr (mgr.14195) 1996 : cluster 0 pgmap v1667: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:19.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:19 smithi153 bash[14340]: audit 2024-03-12T03:04:18.054301+0000 mon.smithi153 (mon.0) 1291 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:04:19.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:19 smithi153 bash[14340]: audit 2024-03-12T03:04:18.062090+0000 mon.smithi153 (mon.0) 1292 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:04:19.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:19 smithi153 bash[14340]: audit 2024-03-12T03:04:18.180712+0000 mgr.smithi153.asomxr (mgr.14195) 1997 : audit 0 from='client.15626 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:19.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:19 smithi153 bash[14340]: audit 2024-03-12T03:04:18.531004+0000 mon.smithi153 (mon.0) 1293 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:04:19.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:19 smithi153 bash[14340]: audit 2024-03-12T03:04:18.532305+0000 mon.smithi153 (mon.0) 1294 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:04:19.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:19 smithi153 bash[14340]: audit 2024-03-12T03:04:18.539437+0000 mon.smithi153 (mon.0) 1295 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:04:19.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:19 smithi153 bash[14340]: audit 2024-03-12T03:04:18.542784+0000 mon.smithi153 (mon.0) 1296 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:04:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:19 smithi163 bash[22347]: cluster 2024-03-12T03:04:17.682168+0000 mgr.smithi153.asomxr (mgr.14195) 1996 : cluster 0 pgmap v1667: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:19 smithi163 bash[22347]: audit 2024-03-12T03:04:18.054301+0000 mon.smithi153 (mon.0) 1291 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:04:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:19 smithi163 bash[22347]: audit 2024-03-12T03:04:18.062090+0000 mon.smithi153 (mon.0) 1292 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:04:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:19 smithi163 bash[22347]: audit 2024-03-12T03:04:18.180712+0000 mgr.smithi153.asomxr (mgr.14195) 1997 : audit 0 from='client.15626 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:19 smithi163 bash[22347]: audit 2024-03-12T03:04:18.531004+0000 mon.smithi153 (mon.0) 1293 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:04:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:19 smithi163 bash[22347]: audit 2024-03-12T03:04:18.532305+0000 mon.smithi153 (mon.0) 1294 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:04:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:19 smithi163 bash[22347]: audit 2024-03-12T03:04:18.539437+0000 mon.smithi153 (mon.0) 1295 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:04:19.484 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:19 smithi163 bash[22347]: audit 2024-03-12T03:04:18.542784+0000 mon.smithi153 (mon.0) 1296 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:04:19.980 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:04:20.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:20 smithi153 bash[14340]: cluster 2024-03-12T03:04:18.534468+0000 mgr.smithi153.asomxr (mgr.14195) 1998 : cluster 0 pgmap v1668: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:20.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:20 smithi153 bash[14340]: cluster 2024-03-12T03:04:18.534796+0000 mgr.smithi153.asomxr (mgr.14195) 1999 : cluster 0 pgmap v1669: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:20 smithi163 bash[22347]: cluster 2024-03-12T03:04:18.534468+0000 mgr.smithi153.asomxr (mgr.14195) 1998 : cluster 0 pgmap v1668: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:20.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:20 smithi163 bash[22347]: cluster 2024-03-12T03:04:18.534796+0000 mgr.smithi153.asomxr (mgr.14195) 1999 : cluster 0 pgmap v1669: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:22.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:22 smithi153 bash[14340]: cluster 2024-03-12T03:04:20.535403+0000 mgr.smithi153.asomxr (mgr.14195) 2000 : cluster 0 pgmap v1670: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:22.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:22 smithi163 bash[22347]: cluster 2024-03-12T03:04:20.535403+0000 mgr.smithi153.asomxr (mgr.14195) 2000 : cluster 0 pgmap v1670: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:24.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:24 smithi153 bash[14340]: cluster 2024-03-12T03:04:22.535984+0000 mgr.smithi153.asomxr (mgr.14195) 2001 : cluster 0 pgmap v1671: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:24 smithi163 bash[22347]: cluster 2024-03-12T03:04:22.535984+0000 mgr.smithi153.asomxr (mgr.14195) 2001 : cluster 0 pgmap v1671: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:24.736 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:04:26.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:26 smithi153 bash[14340]: cluster 2024-03-12T03:04:24.536602+0000 mgr.smithi153.asomxr (mgr.14195) 2002 : cluster 0 pgmap v1672: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:26 smithi163 bash[22347]: cluster 2024-03-12T03:04:24.536602+0000 mgr.smithi153.asomxr (mgr.14195) 2002 : cluster 0 pgmap v1672: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:28.042 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:04:28.043 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:04:28.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:28 smithi153 bash[14340]: cluster 2024-03-12T03:04:26.537144+0000 mgr.smithi153.asomxr (mgr.14195) 2003 : cluster 0 pgmap v1673: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:28 smithi163 bash[22347]: cluster 2024-03-12T03:04:26.537144+0000 mgr.smithi153.asomxr (mgr.14195) 2003 : cluster 0 pgmap v1673: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:28.848 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:04:29.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:29 smithi153 bash[14340]: audit 2024-03-12T03:04:28.031627+0000 mgr.smithi153.asomxr (mgr.14195) 2004 : audit 0 from='client.15630 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:29.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:29 smithi163 bash[22347]: audit 2024-03-12T03:04:28.031627+0000 mgr.smithi153.asomxr (mgr.14195) 2004 : audit 0 from='client.15630 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:29.848 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:04:30.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:30 smithi153 bash[14340]: cluster 2024-03-12T03:04:28.537666+0000 mgr.smithi153.asomxr (mgr.14195) 2005 : cluster 0 pgmap v1674: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:30.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:30 smithi163 bash[22347]: cluster 2024-03-12T03:04:28.537666+0000 mgr.smithi153.asomxr (mgr.14195) 2005 : cluster 0 pgmap v1674: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:31.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:31 smithi163 bash[22347]: audit 2024-03-12T03:04:30.462584+0000 mon.smithi153 (mon.0) 1297 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:04:31.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:31 smithi153 bash[14340]: audit 2024-03-12T03:04:30.462584+0000 mon.smithi153 (mon.0) 1297 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:04:32.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:32 smithi163 bash[22347]: cluster 2024-03-12T03:04:30.538304+0000 mgr.smithi153.asomxr (mgr.14195) 2006 : cluster 0 pgmap v1675: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:32.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:32 smithi153 bash[14340]: cluster 2024-03-12T03:04:30.538304+0000 mgr.smithi153.asomxr (mgr.14195) 2006 : cluster 0 pgmap v1675: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:34.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:34 smithi163 bash[22347]: cluster 2024-03-12T03:04:32.538934+0000 mgr.smithi153.asomxr (mgr.14195) 2007 : cluster 0 pgmap v1676: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:34.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:34 smithi153 bash[14340]: cluster 2024-03-12T03:04:32.538934+0000 mgr.smithi153.asomxr (mgr.14195) 2007 : cluster 0 pgmap v1676: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:34.602 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:04:36.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:36 smithi153 bash[14340]: cluster 2024-03-12T03:04:34.539589+0000 mgr.smithi153.asomxr (mgr.14195) 2008 : cluster 0 pgmap v1677: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:36.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:36 smithi163 bash[22347]: cluster 2024-03-12T03:04:34.539589+0000 mgr.smithi153.asomxr (mgr.14195) 2008 : cluster 0 pgmap v1677: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:38.022 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:04:38.022 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:04:38.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:38 smithi153 bash[14340]: cluster 2024-03-12T03:04:36.540176+0000 mgr.smithi153.asomxr (mgr.14195) 2009 : cluster 0 pgmap v1678: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:38.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:38 smithi163 bash[22347]: cluster 2024-03-12T03:04:36.540176+0000 mgr.smithi153.asomxr (mgr.14195) 2009 : cluster 0 pgmap v1678: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:38.785 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:04:39.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:39 smithi163 bash[22347]: audit 2024-03-12T03:04:38.008068+0000 mgr.smithi153.asomxr (mgr.14195) 2010 : audit 0 from='client.15634 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:39.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:39 smithi153 bash[14340]: audit 2024-03-12T03:04:38.008068+0000 mgr.smithi153.asomxr (mgr.14195) 2010 : audit 0 from='client.15634 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:39.785 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:04:40.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:40 smithi163 bash[22347]: cluster 2024-03-12T03:04:38.540664+0000 mgr.smithi153.asomxr (mgr.14195) 2011 : cluster 0 pgmap v1679: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:40.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:40 smithi153 bash[14340]: cluster 2024-03-12T03:04:38.540664+0000 mgr.smithi153.asomxr (mgr.14195) 2011 : cluster 0 pgmap v1679: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:42.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:42 smithi163 bash[22347]: cluster 2024-03-12T03:04:40.541311+0000 mgr.smithi153.asomxr (mgr.14195) 2012 : cluster 0 pgmap v1680: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:42.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:42 smithi153 bash[14340]: cluster 2024-03-12T03:04:40.541311+0000 mgr.smithi153.asomxr (mgr.14195) 2012 : cluster 0 pgmap v1680: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:44.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:44 smithi163 bash[22347]: cluster 2024-03-12T03:04:42.541851+0000 mgr.smithi153.asomxr (mgr.14195) 2013 : cluster 0 pgmap v1681: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:44.538 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:04:44.565 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:44 smithi153 bash[14340]: cluster 2024-03-12T03:04:42.541851+0000 mgr.smithi153.asomxr (mgr.14195) 2013 : cluster 0 pgmap v1681: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:46.441 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:46 smithi153 bash[14340]: cluster 2024-03-12T03:04:44.542442+0000 mgr.smithi153.asomxr (mgr.14195) 2014 : cluster 0 pgmap v1682: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:46.441 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:46 smithi153 bash[14340]: audit 2024-03-12T03:04:45.462788+0000 mon.smithi153 (mon.0) 1298 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:04:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:46 smithi163 bash[22347]: cluster 2024-03-12T03:04:44.542442+0000 mgr.smithi153.asomxr (mgr.14195) 2014 : cluster 0 pgmap v1682: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:46.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:46 smithi163 bash[22347]: audit 2024-03-12T03:04:45.462788+0000 mon.smithi153 (mon.0) 1298 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:04:47.741 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:04:47.741 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:04:48.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:48 smithi153 bash[14340]: cluster 2024-03-12T03:04:46.542999+0000 mgr.smithi153.asomxr (mgr.14195) 2015 : cluster 0 pgmap v1683: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:48.452 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:04:48.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:48 smithi163 bash[22347]: cluster 2024-03-12T03:04:46.542999+0000 mgr.smithi153.asomxr (mgr.14195) 2015 : cluster 0 pgmap v1683: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:49.453 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:04:49.465 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:49 smithi153 bash[14340]: audit 2024-03-12T03:04:47.729202+0000 mgr.smithi153.asomxr (mgr.14195) 2016 : audit 0 from='client.15638 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:49.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:49 smithi163 bash[22347]: audit 2024-03-12T03:04:47.729202+0000 mgr.smithi153.asomxr (mgr.14195) 2016 : audit 0 from='client.15638 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:50.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:50 smithi163 bash[22347]: cluster 2024-03-12T03:04:48.543401+0000 mgr.smithi153.asomxr (mgr.14195) 2017 : cluster 0 pgmap v1684: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:50.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:50 smithi153 bash[14340]: cluster 2024-03-12T03:04:48.543401+0000 mgr.smithi153.asomxr (mgr.14195) 2017 : cluster 0 pgmap v1684: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:52.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:52 smithi163 bash[22347]: cluster 2024-03-12T03:04:50.544000+0000 mgr.smithi153.asomxr (mgr.14195) 2018 : cluster 0 pgmap v1685: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:52.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:52 smithi153 bash[14340]: cluster 2024-03-12T03:04:50.544000+0000 mgr.smithi153.asomxr (mgr.14195) 2018 : cluster 0 pgmap v1685: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:54.211 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:04:54.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:54 smithi163 bash[22347]: cluster 2024-03-12T03:04:52.544598+0000 mgr.smithi153.asomxr (mgr.14195) 2019 : cluster 0 pgmap v1686: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:54.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:54 smithi153 bash[14340]: cluster 2024-03-12T03:04:52.544598+0000 mgr.smithi153.asomxr (mgr.14195) 2019 : cluster 0 pgmap v1686: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:56.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:56 smithi163 bash[22347]: cluster 2024-03-12T03:04:54.545006+0000 mgr.smithi153.asomxr (mgr.14195) 2020 : cluster 0 pgmap v1687: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:56.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:56 smithi153 bash[14340]: cluster 2024-03-12T03:04:54.545006+0000 mgr.smithi153.asomxr (mgr.14195) 2020 : cluster 0 pgmap v1687: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:57.598 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:04:57.598 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:04:58.358 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:04:58.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:58 smithi163 bash[22347]: cluster 2024-03-12T03:04:56.545610+0000 mgr.smithi153.asomxr (mgr.14195) 2021 : cluster 0 pgmap v1688: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:58.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:58 smithi153 bash[14340]: cluster 2024-03-12T03:04:56.545610+0000 mgr.smithi153.asomxr (mgr.14195) 2021 : cluster 0 pgmap v1688: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:04:59.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:04:59 smithi163 bash[22347]: audit 2024-03-12T03:04:57.582802+0000 mgr.smithi153.asomxr (mgr.14195) 2022 : audit 0 from='client.15642 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:04:59.359 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:04:59.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:04:59 smithi153 bash[14340]: audit 2024-03-12T03:04:57.582802+0000 mgr.smithi153.asomxr (mgr.14195) 2022 : audit 0 from='client.15642 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:00.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:00 smithi163 bash[22347]: cluster 2024-03-12T03:04:58.546213+0000 mgr.smithi153.asomxr (mgr.14195) 2023 : cluster 0 pgmap v1689: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:00.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:00 smithi153 bash[14340]: cluster 2024-03-12T03:04:58.546213+0000 mgr.smithi153.asomxr (mgr.14195) 2023 : cluster 0 pgmap v1689: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:01.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:01 smithi153 bash[14340]: audit 2024-03-12T03:05:00.463419+0000 mon.smithi153 (mon.0) 1299 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:05:01.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:01 smithi163 bash[22347]: audit 2024-03-12T03:05:00.463419+0000 mon.smithi153 (mon.0) 1299 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:05:02.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:02 smithi163 bash[22347]: cluster 2024-03-12T03:05:00.546738+0000 mgr.smithi153.asomxr (mgr.14195) 2024 : cluster 0 pgmap v1690: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:02.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:02 smithi153 bash[14340]: cluster 2024-03-12T03:05:00.546738+0000 mgr.smithi153.asomxr (mgr.14195) 2024 : cluster 0 pgmap v1690: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:04.113 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:05:04.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:04 smithi163 bash[22347]: cluster 2024-03-12T03:05:02.547331+0000 mgr.smithi153.asomxr (mgr.14195) 2025 : cluster 0 pgmap v1691: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:04.574 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:04 smithi153 bash[14340]: cluster 2024-03-12T03:05:02.547331+0000 mgr.smithi153.asomxr (mgr.14195) 2025 : cluster 0 pgmap v1691: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:06.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:06 smithi163 bash[22347]: cluster 2024-03-12T03:05:04.547961+0000 mgr.smithi153.asomxr (mgr.14195) 2026 : cluster 0 pgmap v1692: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:06.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:06 smithi153 bash[14340]: cluster 2024-03-12T03:05:04.547961+0000 mgr.smithi153.asomxr (mgr.14195) 2026 : cluster 0 pgmap v1692: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:07.395 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:05:07.395 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:05:08.191 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:05:08.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:08 smithi163 bash[22347]: cluster 2024-03-12T03:05:06.548527+0000 mgr.smithi153.asomxr (mgr.14195) 2027 : cluster 0 pgmap v1693: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:08.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:08 smithi163 bash[22347]: audit 2024-03-12T03:05:07.378146+0000 mgr.smithi153.asomxr (mgr.14195) 2028 : audit 0 from='client.15646 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:08.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:08 smithi153 bash[14340]: cluster 2024-03-12T03:05:06.548527+0000 mgr.smithi153.asomxr (mgr.14195) 2027 : cluster 0 pgmap v1693: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:08.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:08 smithi153 bash[14340]: audit 2024-03-12T03:05:07.378146+0000 mgr.smithi153.asomxr (mgr.14195) 2028 : audit 0 from='client.15646 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:09.192 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:05:09.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:09 smithi153 bash[14340]: cluster 2024-03-12T03:05:08.549166+0000 mgr.smithi153.asomxr (mgr.14195) 2029 : cluster 0 pgmap v1694: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:09.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:09 smithi163 bash[22347]: cluster 2024-03-12T03:05:08.549166+0000 mgr.smithi153.asomxr (mgr.14195) 2029 : cluster 0 pgmap v1694: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:11.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:11 smithi153 bash[14340]: cluster 2024-03-12T03:05:10.549794+0000 mgr.smithi153.asomxr (mgr.14195) 2030 : cluster 0 pgmap v1695: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:11.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:11 smithi163 bash[22347]: cluster 2024-03-12T03:05:10.549794+0000 mgr.smithi153.asomxr (mgr.14195) 2030 : cluster 0 pgmap v1695: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:13.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:13 smithi153 bash[14340]: cluster 2024-03-12T03:05:12.550437+0000 mgr.smithi153.asomxr (mgr.14195) 2031 : cluster 0 pgmap v1696: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:13.941 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:05:13.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:13 smithi163 bash[22347]: cluster 2024-03-12T03:05:12.550437+0000 mgr.smithi153.asomxr (mgr.14195) 2031 : cluster 0 pgmap v1696: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:15.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:15 smithi153 bash[14340]: cluster 2024-03-12T03:05:14.551093+0000 mgr.smithi153.asomxr (mgr.14195) 2032 : cluster 0 pgmap v1697: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:15.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:15 smithi153 bash[14340]: audit 2024-03-12T03:05:15.462974+0000 mon.smithi153 (mon.0) 1300 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:05:15.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:15 smithi163 bash[22347]: cluster 2024-03-12T03:05:14.551093+0000 mgr.smithi153.asomxr (mgr.14195) 2032 : cluster 0 pgmap v1697: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:15.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:15 smithi163 bash[22347]: audit 2024-03-12T03:05:15.462974+0000 mon.smithi153 (mon.0) 1300 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:05:17.750 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:05:17.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:17 smithi163 bash[22347]: cluster 2024-03-12T03:05:16.551667+0000 mgr.smithi153.asomxr (mgr.14195) 2033 : cluster 0 pgmap v1698: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:17.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:17 smithi163 bash[22347]: audit 2024-03-12T03:05:16.987318+0000 mgr.smithi153.asomxr (mgr.14195) 2034 : audit 0 from='client.15650 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:18.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:17 smithi153 bash[14340]: cluster 2024-03-12T03:05:16.551667+0000 mgr.smithi153.asomxr (mgr.14195) 2033 : cluster 0 pgmap v1698: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:18.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:17 smithi153 bash[14340]: audit 2024-03-12T03:05:16.987318+0000 mgr.smithi153.asomxr (mgr.14195) 2034 : audit 0 from='client.15650 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:18.752 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:05:19.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:19 smithi163 bash[22347]: cluster 2024-03-12T03:05:18.552382+0000 mgr.smithi153.asomxr (mgr.14195) 2035 : cluster 0 pgmap v1699: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:19.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:19 smithi163 bash[22347]: audit 2024-03-12T03:05:18.626945+0000 mon.smithi153 (mon.0) 1301 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:05:19.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:19 smithi163 bash[22347]: audit 2024-03-12T03:05:19.213982+0000 mon.smithi153 (mon.0) 1302 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:05:19.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:19 smithi163 bash[22347]: audit 2024-03-12T03:05:19.216103+0000 mon.smithi153 (mon.0) 1303 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:05:19.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:19 smithi163 bash[22347]: cluster 2024-03-12T03:05:19.218820+0000 mgr.smithi153.asomxr (mgr.14195) 2036 : cluster 0 pgmap v1700: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:19.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:19 smithi163 bash[22347]: cluster 2024-03-12T03:05:19.219211+0000 mgr.smithi153.asomxr (mgr.14195) 2037 : cluster 0 pgmap v1701: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:19.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:19 smithi163 bash[22347]: audit 2024-03-12T03:05:19.225162+0000 mon.smithi153 (mon.0) 1304 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:05:19.734 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:19 smithi163 bash[22347]: audit 2024-03-12T03:05:19.230616+0000 mon.smithi153 (mon.0) 1305 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:05:19.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:19 smithi153 bash[14340]: cluster 2024-03-12T03:05:18.552382+0000 mgr.smithi153.asomxr (mgr.14195) 2035 : cluster 0 pgmap v1699: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:19.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:19 smithi153 bash[14340]: audit 2024-03-12T03:05:18.626945+0000 mon.smithi153 (mon.0) 1301 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:05:19.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:19 smithi153 bash[14340]: audit 2024-03-12T03:05:19.213982+0000 mon.smithi153 (mon.0) 1302 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:05:19.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:19 smithi153 bash[14340]: audit 2024-03-12T03:05:19.216103+0000 mon.smithi153 (mon.0) 1303 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:05:19.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:19 smithi153 bash[14340]: cluster 2024-03-12T03:05:19.218820+0000 mgr.smithi153.asomxr (mgr.14195) 2036 : cluster 0 pgmap v1700: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:19.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:19 smithi153 bash[14340]: cluster 2024-03-12T03:05:19.219211+0000 mgr.smithi153.asomxr (mgr.14195) 2037 : cluster 0 pgmap v1701: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:19.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:19 smithi153 bash[14340]: audit 2024-03-12T03:05:19.225162+0000 mon.smithi153 (mon.0) 1304 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:05:19.824 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:19 smithi153 bash[14340]: audit 2024-03-12T03:05:19.230616+0000 mon.smithi153 (mon.0) 1305 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:05:22.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:22 smithi153 bash[14340]: cluster 2024-03-12T03:05:21.219806+0000 mgr.smithi153.asomxr (mgr.14195) 2038 : cluster 0 pgmap v1702: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:22.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:22 smithi163 bash[22347]: cluster 2024-03-12T03:05:21.219806+0000 mgr.smithi153.asomxr (mgr.14195) 2038 : cluster 0 pgmap v1702: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:23.497 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:05:24.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:24 smithi153 bash[14340]: cluster 2024-03-12T03:05:23.220558+0000 mgr.smithi153.asomxr (mgr.14195) 2039 : cluster 0 pgmap v1703: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:24.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:24 smithi163 bash[22347]: cluster 2024-03-12T03:05:23.220558+0000 mgr.smithi153.asomxr (mgr.14195) 2039 : cluster 0 pgmap v1703: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:26.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:26 smithi153 bash[14340]: cluster 2024-03-12T03:05:25.221278+0000 mgr.smithi153.asomxr (mgr.14195) 2040 : cluster 0 pgmap v1704: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:26.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:26 smithi163 bash[22347]: cluster 2024-03-12T03:05:25.221278+0000 mgr.smithi153.asomxr (mgr.14195) 2040 : cluster 0 pgmap v1704: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:26.830 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:05:26.830 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:05:27.592 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:05:28.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:28 smithi153 bash[14340]: audit 2024-03-12T03:05:26.820281+0000 mgr.smithi153.asomxr (mgr.14195) 2041 : audit 0 from='client.15654 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:28.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:28 smithi153 bash[14340]: cluster 2024-03-12T03:05:27.221860+0000 mgr.smithi153.asomxr (mgr.14195) 2042 : cluster 0 pgmap v1705: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:28.593 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:05:28.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:28 smithi163 bash[22347]: audit 2024-03-12T03:05:26.820281+0000 mgr.smithi153.asomxr (mgr.14195) 2041 : audit 0 from='client.15654 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:28.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:28 smithi163 bash[22347]: cluster 2024-03-12T03:05:27.221860+0000 mgr.smithi153.asomxr (mgr.14195) 2042 : cluster 0 pgmap v1705: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:29.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:29 smithi153 bash[14340]: cluster 2024-03-12T03:05:29.222484+0000 mgr.smithi153.asomxr (mgr.14195) 2043 : cluster 0 pgmap v1706: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:29.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:29 smithi163 bash[22347]: cluster 2024-03-12T03:05:29.222484+0000 mgr.smithi153.asomxr (mgr.14195) 2043 : cluster 0 pgmap v1706: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:30.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:30 smithi153 bash[14340]: audit 2024-03-12T03:05:30.464058+0000 mon.smithi153 (mon.0) 1306 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:05:30.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:30 smithi163 bash[22347]: audit 2024-03-12T03:05:30.464058+0000 mon.smithi153 (mon.0) 1306 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:05:31.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:31 smithi163 bash[22347]: cluster 2024-03-12T03:05:31.223067+0000 mgr.smithi153.asomxr (mgr.14195) 2044 : cluster 0 pgmap v1707: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:32.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:31 smithi153 bash[14340]: cluster 2024-03-12T03:05:31.223067+0000 mgr.smithi153.asomxr (mgr.14195) 2044 : cluster 0 pgmap v1707: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:33.343 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:05:34.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:34 smithi153 bash[14340]: cluster 2024-03-12T03:05:33.223678+0000 mgr.smithi153.asomxr (mgr.14195) 2045 : cluster 0 pgmap v1708: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:34.670 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:34 smithi163 bash[22347]: cluster 2024-03-12T03:05:33.223678+0000 mgr.smithi153.asomxr (mgr.14195) 2045 : cluster 0 pgmap v1708: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:36.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:36 smithi153 bash[14340]: cluster 2024-03-12T03:05:35.224312+0000 mgr.smithi153.asomxr (mgr.14195) 2046 : cluster 0 pgmap v1709: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:36.678 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:05:36.679 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:05:36.732 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:36 smithi163 bash[22347]: cluster 2024-03-12T03:05:35.224312+0000 mgr.smithi153.asomxr (mgr.14195) 2046 : cluster 0 pgmap v1709: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:37.448 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:05:38.449 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:05:38.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:38 smithi153 bash[14340]: audit 2024-03-12T03:05:36.665005+0000 mgr.smithi153.asomxr (mgr.14195) 2047 : audit 0 from='client.15658 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:38.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:38 smithi153 bash[14340]: cluster 2024-03-12T03:05:37.224914+0000 mgr.smithi153.asomxr (mgr.14195) 2048 : cluster 0 pgmap v1710: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:38.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:38 smithi163 bash[22347]: audit 2024-03-12T03:05:36.665005+0000 mgr.smithi153.asomxr (mgr.14195) 2047 : audit 0 from='client.15658 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:38.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:38 smithi163 bash[22347]: cluster 2024-03-12T03:05:37.224914+0000 mgr.smithi153.asomxr (mgr.14195) 2048 : cluster 0 pgmap v1710: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:39.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:39 smithi153 bash[14340]: cluster 2024-03-12T03:05:39.225559+0000 mgr.smithi153.asomxr (mgr.14195) 2049 : cluster 0 pgmap v1711: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:39.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:39 smithi163 bash[22347]: cluster 2024-03-12T03:05:39.225559+0000 mgr.smithi153.asomxr (mgr.14195) 2049 : cluster 0 pgmap v1711: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:42.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:42 smithi153 bash[14340]: cluster 2024-03-12T03:05:41.226271+0000 mgr.smithi153.asomxr (mgr.14195) 2050 : cluster 0 pgmap v1712: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:42 smithi163 bash[22347]: cluster 2024-03-12T03:05:41.226271+0000 mgr.smithi153.asomxr (mgr.14195) 2050 : cluster 0 pgmap v1712: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:43.204 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:05:44.522 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:44 smithi153 bash[14340]: cluster 2024-03-12T03:05:43.227030+0000 mgr.smithi153.asomxr (mgr.14195) 2051 : cluster 0 pgmap v1713: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:44.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:44 smithi163 bash[22347]: cluster 2024-03-12T03:05:43.227030+0000 mgr.smithi153.asomxr (mgr.14195) 2051 : cluster 0 pgmap v1713: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:46.415 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:05:46.415 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:05:46.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:46 smithi153 bash[14340]: cluster 2024-03-12T03:05:45.227701+0000 mgr.smithi153.asomxr (mgr.14195) 2052 : cluster 0 pgmap v1714: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:46.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:46 smithi153 bash[14340]: audit 2024-03-12T03:05:45.463773+0000 mon.smithi153 (mon.0) 1307 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:05:46.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:46 smithi163 bash[22347]: cluster 2024-03-12T03:05:45.227701+0000 mgr.smithi153.asomxr (mgr.14195) 2052 : cluster 0 pgmap v1714: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:46.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:46 smithi163 bash[22347]: audit 2024-03-12T03:05:45.463773+0000 mon.smithi153 (mon.0) 1307 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:05:47.243 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:05:47.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:47 smithi153 bash[14340]: audit 2024-03-12T03:05:46.400423+0000 mgr.smithi153.asomxr (mgr.14195) 2053 : audit 0 from='client.15662 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:47.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:47 smithi163 bash[22347]: audit 2024-03-12T03:05:46.400423+0000 mgr.smithi153.asomxr (mgr.14195) 2053 : audit 0 from='client.15662 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:48.244 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:05:48.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:48 smithi153 bash[14340]: cluster 2024-03-12T03:05:47.228335+0000 mgr.smithi153.asomxr (mgr.14195) 2054 : cluster 0 pgmap v1715: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:48.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:48 smithi163 bash[22347]: cluster 2024-03-12T03:05:47.228335+0000 mgr.smithi153.asomxr (mgr.14195) 2054 : cluster 0 pgmap v1715: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:49.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:49 smithi153 bash[14340]: cluster 2024-03-12T03:05:49.228927+0000 mgr.smithi153.asomxr (mgr.14195) 2055 : cluster 0 pgmap v1716: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:49.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:49 smithi163 bash[22347]: cluster 2024-03-12T03:05:49.228927+0000 mgr.smithi153.asomxr (mgr.14195) 2055 : cluster 0 pgmap v1716: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:52.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:52 smithi153 bash[14340]: cluster 2024-03-12T03:05:51.229669+0000 mgr.smithi153.asomxr (mgr.14195) 2056 : cluster 0 pgmap v1717: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:52.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:52 smithi163 bash[22347]: cluster 2024-03-12T03:05:51.229669+0000 mgr.smithi153.asomxr (mgr.14195) 2056 : cluster 0 pgmap v1717: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:52.998 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:05:54.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:54 smithi153 bash[14340]: cluster 2024-03-12T03:05:53.230221+0000 mgr.smithi153.asomxr (mgr.14195) 2057 : cluster 0 pgmap v1718: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:54.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:54 smithi163 bash[22347]: cluster 2024-03-12T03:05:53.230221+0000 mgr.smithi153.asomxr (mgr.14195) 2057 : cluster 0 pgmap v1718: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:56.391 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:05:56.392 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:05:56.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:56 smithi153 bash[14340]: cluster 2024-03-12T03:05:55.230903+0000 mgr.smithi153.asomxr (mgr.14195) 2058 : cluster 0 pgmap v1719: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:56.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:56 smithi163 bash[22347]: cluster 2024-03-12T03:05:55.230903+0000 mgr.smithi153.asomxr (mgr.14195) 2058 : cluster 0 pgmap v1719: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:57.221 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:05:57.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:57 smithi153 bash[14340]: audit 2024-03-12T03:05:56.376820+0000 mgr.smithi153.asomxr (mgr.14195) 2059 : audit 0 from='client.15666 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:57.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:57 smithi163 bash[22347]: audit 2024-03-12T03:05:56.376820+0000 mgr.smithi153.asomxr (mgr.14195) 2059 : audit 0 from='client.15666 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:05:58.222 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:05:58.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:58 smithi153 bash[14340]: cluster 2024-03-12T03:05:57.231563+0000 mgr.smithi153.asomxr (mgr.14195) 2060 : cluster 0 pgmap v1720: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:58.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:58 smithi163 bash[22347]: cluster 2024-03-12T03:05:57.231563+0000 mgr.smithi153.asomxr (mgr.14195) 2060 : cluster 0 pgmap v1720: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:59.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:05:59 smithi153 bash[14340]: cluster 2024-03-12T03:05:59.232284+0000 mgr.smithi153.asomxr (mgr.14195) 2061 : cluster 0 pgmap v1721: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:05:59.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:05:59 smithi163 bash[22347]: cluster 2024-03-12T03:05:59.232284+0000 mgr.smithi153.asomxr (mgr.14195) 2061 : cluster 0 pgmap v1721: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:00.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:00 smithi163 bash[22347]: audit 2024-03-12T03:06:00.464817+0000 mon.smithi153 (mon.0) 1308 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:06:01.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:00 smithi153 bash[14340]: audit 2024-03-12T03:06:00.464817+0000 mon.smithi153 (mon.0) 1308 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:06:01.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:01 smithi163 bash[22347]: cluster 2024-03-12T03:06:01.232973+0000 mgr.smithi153.asomxr (mgr.14195) 2062 : cluster 0 pgmap v1722: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:02.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:01 smithi153 bash[14340]: cluster 2024-03-12T03:06:01.232973+0000 mgr.smithi153.asomxr (mgr.14195) 2062 : cluster 0 pgmap v1722: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:02.977 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:06:04.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:04 smithi153 bash[14340]: cluster 2024-03-12T03:06:03.233611+0000 mgr.smithi153.asomxr (mgr.14195) 2063 : cluster 0 pgmap v1723: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:04.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:04 smithi163 bash[22347]: cluster 2024-03-12T03:06:03.233611+0000 mgr.smithi153.asomxr (mgr.14195) 2063 : cluster 0 pgmap v1723: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:06.270 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:06:06.270 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:06:06.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:06 smithi153 bash[14340]: cluster 2024-03-12T03:06:05.234182+0000 mgr.smithi153.asomxr (mgr.14195) 2064 : cluster 0 pgmap v1724: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:06.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:06 smithi163 bash[22347]: cluster 2024-03-12T03:06:05.234182+0000 mgr.smithi153.asomxr (mgr.14195) 2064 : cluster 0 pgmap v1724: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:07.036 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:06:07.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:07 smithi153 bash[14340]: audit 2024-03-12T03:06:06.255910+0000 mgr.smithi153.asomxr (mgr.14195) 2065 : audit 0 from='client.15670 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:07.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:07 smithi163 bash[22347]: audit 2024-03-12T03:06:06.255910+0000 mgr.smithi153.asomxr (mgr.14195) 2065 : audit 0 from='client.15670 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:08.037 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:06:08.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:08 smithi153 bash[14340]: cluster 2024-03-12T03:06:07.234817+0000 mgr.smithi153.asomxr (mgr.14195) 2066 : cluster 0 pgmap v1725: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:08.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:08 smithi163 bash[22347]: cluster 2024-03-12T03:06:07.234817+0000 mgr.smithi153.asomxr (mgr.14195) 2066 : cluster 0 pgmap v1725: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:09.934 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:09 smithi153 bash[14340]: cluster 2024-03-12T03:06:09.235410+0000 mgr.smithi153.asomxr (mgr.14195) 2067 : cluster 0 pgmap v1726: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:09.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:09 smithi163 bash[22347]: cluster 2024-03-12T03:06:09.235410+0000 mgr.smithi153.asomxr (mgr.14195) 2067 : cluster 0 pgmap v1726: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:12.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:12 smithi153 bash[14340]: cluster 2024-03-12T03:06:11.236061+0000 mgr.smithi153.asomxr (mgr.14195) 2068 : cluster 0 pgmap v1727: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:12.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:12 smithi163 bash[22347]: cluster 2024-03-12T03:06:11.236061+0000 mgr.smithi153.asomxr (mgr.14195) 2068 : cluster 0 pgmap v1727: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:12.792 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:06:14.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:14 smithi153 bash[14340]: cluster 2024-03-12T03:06:13.236656+0000 mgr.smithi153.asomxr (mgr.14195) 2069 : cluster 0 pgmap v1728: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:14.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:14 smithi163 bash[22347]: cluster 2024-03-12T03:06:13.236656+0000 mgr.smithi153.asomxr (mgr.14195) 2069 : cluster 0 pgmap v1728: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:16.173 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:06:16.173 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:06:16.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:16 smithi153 bash[14340]: cluster 2024-03-12T03:06:15.237295+0000 mgr.smithi153.asomxr (mgr.14195) 2070 : cluster 0 pgmap v1729: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:16.573 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:16 smithi153 bash[14340]: audit 2024-03-12T03:06:15.465267+0000 mon.smithi153 (mon.0) 1309 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:06:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:16 smithi163 bash[22347]: cluster 2024-03-12T03:06:15.237295+0000 mgr.smithi153.asomxr (mgr.14195) 2070 : cluster 0 pgmap v1729: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:16.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:16 smithi163 bash[22347]: audit 2024-03-12T03:06:15.465267+0000 mon.smithi153 (mon.0) 1309 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:06:16.979 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:06:17.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:17 smithi153 bash[14340]: audit 2024-03-12T03:06:16.158953+0000 mgr.smithi153.asomxr (mgr.14195) 2071 : audit 0 from='client.15674 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:17.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:17 smithi163 bash[22347]: audit 2024-03-12T03:06:16.158953+0000 mgr.smithi153.asomxr (mgr.14195) 2071 : audit 0 from='client.15674 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:17.980 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:06:18.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:18 smithi153 bash[14340]: cluster 2024-03-12T03:06:17.237936+0000 mgr.smithi153.asomxr (mgr.14195) 2072 : cluster 0 pgmap v1730: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:18.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:18 smithi163 bash[22347]: cluster 2024-03-12T03:06:17.237936+0000 mgr.smithi153.asomxr (mgr.14195) 2072 : cluster 0 pgmap v1730: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:19.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:19 smithi163 bash[22347]: cluster 2024-03-12T03:06:19.238668+0000 mgr.smithi153.asomxr (mgr.14195) 2073 : cluster 0 pgmap v1731: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:19.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:19 smithi163 bash[22347]: audit 2024-03-12T03:06:19.322041+0000 mon.smithi153 (mon.0) 1310 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:06:20.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:19 smithi153 bash[14340]: cluster 2024-03-12T03:06:19.238668+0000 mgr.smithi153.asomxr (mgr.14195) 2073 : cluster 0 pgmap v1731: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:20.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:19 smithi153 bash[14340]: audit 2024-03-12T03:06:19.322041+0000 mon.smithi153 (mon.0) 1310 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:06:20.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:20 smithi163 bash[22347]: audit 2024-03-12T03:06:19.907866+0000 mon.smithi153 (mon.0) 1311 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:06:20.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:20 smithi163 bash[22347]: audit 2024-03-12T03:06:19.909402+0000 mon.smithi153 (mon.0) 1312 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:06:20.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:20 smithi163 bash[22347]: cluster 2024-03-12T03:06:19.911885+0000 mgr.smithi153.asomxr (mgr.14195) 2074 : cluster 0 pgmap v1732: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:20.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:20 smithi163 bash[22347]: audit 2024-03-12T03:06:19.918189+0000 mon.smithi153 (mon.0) 1313 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:06:20.984 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:20 smithi163 bash[22347]: audit 2024-03-12T03:06:19.922946+0000 mon.smithi153 (mon.0) 1314 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:06:21.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:20 smithi153 bash[14340]: audit 2024-03-12T03:06:19.907866+0000 mon.smithi153 (mon.0) 1311 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:06:21.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:20 smithi153 bash[14340]: audit 2024-03-12T03:06:19.909402+0000 mon.smithi153 (mon.0) 1312 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:06:21.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:20 smithi153 bash[14340]: cluster 2024-03-12T03:06:19.911885+0000 mgr.smithi153.asomxr (mgr.14195) 2074 : cluster 0 pgmap v1732: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:21.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:20 smithi153 bash[14340]: audit 2024-03-12T03:06:19.918189+0000 mon.smithi153 (mon.0) 1313 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:06:21.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:20 smithi153 bash[14340]: audit 2024-03-12T03:06:19.922946+0000 mon.smithi153 (mon.0) 1314 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:06:22.728 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:06:23.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:22 smithi153 bash[14340]: cluster 2024-03-12T03:06:21.912543+0000 mgr.smithi153.asomxr (mgr.14195) 2075 : cluster 0 pgmap v1733: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:23.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:22 smithi163 bash[22347]: cluster 2024-03-12T03:06:21.912543+0000 mgr.smithi153.asomxr (mgr.14195) 2075 : cluster 0 pgmap v1733: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:24.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:24 smithi163 bash[22347]: cluster 2024-03-12T03:06:23.913174+0000 mgr.smithi153.asomxr (mgr.14195) 2076 : cluster 0 pgmap v1734: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:25.091 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:24 smithi153 bash[14340]: cluster 2024-03-12T03:06:23.913174+0000 mgr.smithi153.asomxr (mgr.14195) 2076 : cluster 0 pgmap v1734: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:26.189 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:06:26.189 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:06:27.025 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:06:27.232 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:26 smithi163 bash[22347]: cluster 2024-03-12T03:06:25.913799+0000 mgr.smithi153.asomxr (mgr.14195) 2077 : cluster 0 pgmap v1735: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:27.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:26 smithi163 bash[22347]: audit 2024-03-12T03:06:26.173347+0000 mgr.smithi153.asomxr (mgr.14195) 2078 : audit 0 from='client.15678 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:27.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:26 smithi153 bash[14340]: cluster 2024-03-12T03:06:25.913799+0000 mgr.smithi153.asomxr (mgr.14195) 2077 : cluster 0 pgmap v1735: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:27.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:26 smithi153 bash[14340]: audit 2024-03-12T03:06:26.173347+0000 mgr.smithi153.asomxr (mgr.14195) 2078 : audit 0 from='client.15678 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:28.027 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:06:29.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:28 smithi163 bash[22347]: cluster 2024-03-12T03:06:27.914402+0000 mgr.smithi153.asomxr (mgr.14195) 2079 : cluster 0 pgmap v1736: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:29.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:28 smithi153 bash[14340]: cluster 2024-03-12T03:06:27.914402+0000 mgr.smithi153.asomxr (mgr.14195) 2079 : cluster 0 pgmap v1736: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:31.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:30 smithi163 bash[22347]: cluster 2024-03-12T03:06:29.915038+0000 mgr.smithi153.asomxr (mgr.14195) 2080 : cluster 0 pgmap v1737: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:31.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:30 smithi163 bash[22347]: audit 2024-03-12T03:06:30.465699+0000 mon.smithi153 (mon.0) 1315 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:06:31.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:30 smithi153 bash[14340]: cluster 2024-03-12T03:06:29.915038+0000 mgr.smithi153.asomxr (mgr.14195) 2080 : cluster 0 pgmap v1737: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:31.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:30 smithi153 bash[14340]: audit 2024-03-12T03:06:30.465699+0000 mon.smithi153 (mon.0) 1315 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:06:32.782 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:06:33.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:32 smithi153 bash[14340]: cluster 2024-03-12T03:06:31.915686+0000 mgr.smithi153.asomxr (mgr.14195) 2081 : cluster 0 pgmap v1738: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:33.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:32 smithi163 bash[22347]: cluster 2024-03-12T03:06:31.915686+0000 mgr.smithi153.asomxr (mgr.14195) 2081 : cluster 0 pgmap v1738: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:34.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:34 smithi153 bash[14340]: cluster 2024-03-12T03:06:33.916305+0000 mgr.smithi153.asomxr (mgr.14195) 2082 : cluster 0 pgmap v1739: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:34.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:34 smithi163 bash[22347]: cluster 2024-03-12T03:06:33.916305+0000 mgr.smithi153.asomxr (mgr.14195) 2082 : cluster 0 pgmap v1739: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:36.222 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:06:36.222 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:06:37.010 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:06:37.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:36 smithi163 bash[22347]: cluster 2024-03-12T03:06:35.917016+0000 mgr.smithi153.asomxr (mgr.14195) 2083 : cluster 0 pgmap v1740: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:37.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:36 smithi163 bash[22347]: audit 2024-03-12T03:06:36.206656+0000 mgr.smithi153.asomxr (mgr.14195) 2084 : audit 0 from='client.15682 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:37.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:36 smithi153 bash[14340]: cluster 2024-03-12T03:06:35.917016+0000 mgr.smithi153.asomxr (mgr.14195) 2083 : cluster 0 pgmap v1740: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:37.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:36 smithi153 bash[14340]: audit 2024-03-12T03:06:36.206656+0000 mgr.smithi153.asomxr (mgr.14195) 2084 : audit 0 from='client.15682 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:38.011 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:06:39.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:38 smithi163 bash[22347]: cluster 2024-03-12T03:06:37.917680+0000 mgr.smithi153.asomxr (mgr.14195) 2085 : cluster 0 pgmap v1741: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:39.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:38 smithi153 bash[14340]: cluster 2024-03-12T03:06:37.917680+0000 mgr.smithi153.asomxr (mgr.14195) 2085 : cluster 0 pgmap v1741: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:41.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:40 smithi153 bash[14340]: cluster 2024-03-12T03:06:39.918299+0000 mgr.smithi153.asomxr (mgr.14195) 2086 : cluster 0 pgmap v1742: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:41.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:40 smithi163 bash[22347]: cluster 2024-03-12T03:06:39.918299+0000 mgr.smithi153.asomxr (mgr.14195) 2086 : cluster 0 pgmap v1742: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:42.760 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:06:43.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:42 smithi153 bash[14340]: cluster 2024-03-12T03:06:41.918936+0000 mgr.smithi153.asomxr (mgr.14195) 2087 : cluster 0 pgmap v1743: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:43.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:42 smithi163 bash[22347]: cluster 2024-03-12T03:06:41.918936+0000 mgr.smithi153.asomxr (mgr.14195) 2087 : cluster 0 pgmap v1743: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:44.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:44 smithi163 bash[22347]: cluster 2024-03-12T03:06:43.919458+0000 mgr.smithi153.asomxr (mgr.14195) 2088 : cluster 0 pgmap v1744: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:45.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:44 smithi153 bash[14340]: cluster 2024-03-12T03:06:43.919458+0000 mgr.smithi153.asomxr (mgr.14195) 2088 : cluster 0 pgmap v1744: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:45.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:45 smithi163 bash[22347]: audit 2024-03-12T03:06:45.465867+0000 mon.smithi153 (mon.0) 1316 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:06:46.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:45 smithi153 bash[14340]: audit 2024-03-12T03:06:45.465867+0000 mon.smithi153 (mon.0) 1316 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:06:46.184 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:06:46.184 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:06:46.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:46 smithi153 bash[14340]: cluster 2024-03-12T03:06:45.920044+0000 mgr.smithi153.asomxr (mgr.14195) 2089 : cluster 0 pgmap v1745: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:46.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:46 smithi153 bash[14340]: audit 2024-03-12T03:06:46.173912+0000 mgr.smithi153.asomxr (mgr.14195) 2090 : audit 0 from='client.15686 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:46.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:46 smithi163 bash[22347]: cluster 2024-03-12T03:06:45.920044+0000 mgr.smithi153.asomxr (mgr.14195) 2089 : cluster 0 pgmap v1745: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:46.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:46 smithi163 bash[22347]: audit 2024-03-12T03:06:46.173912+0000 mgr.smithi153.asomxr (mgr.14195) 2090 : audit 0 from='client.15686 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:47.105 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:06:48.107 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:06:49.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:48 smithi163 bash[22347]: cluster 2024-03-12T03:06:47.920661+0000 mgr.smithi153.asomxr (mgr.14195) 2091 : cluster 0 pgmap v1746: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:49.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:48 smithi153 bash[14340]: cluster 2024-03-12T03:06:47.920661+0000 mgr.smithi153.asomxr (mgr.14195) 2091 : cluster 0 pgmap v1746: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:51.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:50 smithi153 bash[14340]: cluster 2024-03-12T03:06:49.921259+0000 mgr.smithi153.asomxr (mgr.14195) 2092 : cluster 0 pgmap v1747: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:51.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:50 smithi163 bash[22347]: cluster 2024-03-12T03:06:49.921259+0000 mgr.smithi153.asomxr (mgr.14195) 2092 : cluster 0 pgmap v1747: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:52.861 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:06:53.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:52 smithi153 bash[14340]: cluster 2024-03-12T03:06:51.921856+0000 mgr.smithi153.asomxr (mgr.14195) 2093 : cluster 0 pgmap v1748: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:53.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:52 smithi163 bash[22347]: cluster 2024-03-12T03:06:51.921856+0000 mgr.smithi153.asomxr (mgr.14195) 2093 : cluster 0 pgmap v1748: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:54.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:54 smithi163 bash[22347]: cluster 2024-03-12T03:06:53.922509+0000 mgr.smithi153.asomxr (mgr.14195) 2094 : cluster 0 pgmap v1749: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:55.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:54 smithi153 bash[14340]: cluster 2024-03-12T03:06:53.922509+0000 mgr.smithi153.asomxr (mgr.14195) 2094 : cluster 0 pgmap v1749: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:56.407 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:06:56.407 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:06:57.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:56 smithi163 bash[22347]: cluster 2024-03-12T03:06:55.923183+0000 mgr.smithi153.asomxr (mgr.14195) 2095 : cluster 0 pgmap v1750: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:57.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:56 smithi163 bash[22347]: audit 2024-03-12T03:06:56.391329+0000 mgr.smithi153.asomxr (mgr.14195) 2096 : audit 0 from='client.15690 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:57.268 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:56 smithi153 bash[14340]: cluster 2024-03-12T03:06:55.923183+0000 mgr.smithi153.asomxr (mgr.14195) 2095 : cluster 0 pgmap v1750: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:57.269 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:56 smithi153 bash[14340]: audit 2024-03-12T03:06:56.391329+0000 mgr.smithi153.asomxr (mgr.14195) 2096 : audit 0 from='client.15690 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:06:57.270 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:06:58.271 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:06:59.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:06:58 smithi153 bash[14340]: cluster 2024-03-12T03:06:57.923791+0000 mgr.smithi153.asomxr (mgr.14195) 2097 : cluster 0 pgmap v1751: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:06:59.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:06:58 smithi163 bash[22347]: cluster 2024-03-12T03:06:57.923791+0000 mgr.smithi153.asomxr (mgr.14195) 2097 : cluster 0 pgmap v1751: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:01.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:00 smithi153 bash[14340]: cluster 2024-03-12T03:06:59.924406+0000 mgr.smithi153.asomxr (mgr.14195) 2098 : cluster 0 pgmap v1752: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:01.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:00 smithi153 bash[14340]: audit 2024-03-12T03:07:00.466147+0000 mon.smithi153 (mon.0) 1317 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:07:01.452 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:00 smithi163 bash[22347]: cluster 2024-03-12T03:06:59.924406+0000 mgr.smithi153.asomxr (mgr.14195) 2098 : cluster 0 pgmap v1752: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:01.452 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:00 smithi163 bash[22347]: audit 2024-03-12T03:07:00.466147+0000 mon.smithi153 (mon.0) 1317 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:07:03.024 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:07:03.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:02 smithi153 bash[14340]: cluster 2024-03-12T03:07:01.925048+0000 mgr.smithi153.asomxr (mgr.14195) 2099 : cluster 0 pgmap v1753: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:03.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:02 smithi163 bash[22347]: cluster 2024-03-12T03:07:01.925048+0000 mgr.smithi153.asomxr (mgr.14195) 2099 : cluster 0 pgmap v1753: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:04.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:04 smithi153 bash[14340]: cluster 2024-03-12T03:07:03.925563+0000 mgr.smithi153.asomxr (mgr.14195) 2100 : cluster 0 pgmap v1754: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:04.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:04 smithi163 bash[22347]: cluster 2024-03-12T03:07:03.925563+0000 mgr.smithi153.asomxr (mgr.14195) 2100 : cluster 0 pgmap v1754: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:06.418 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:07:06.418 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:07:07.252 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:06 smithi153 bash[14340]: cluster 2024-03-12T03:07:05.925950+0000 mgr.smithi153.asomxr (mgr.14195) 2101 : cluster 0 pgmap v1755: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:07.252 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:06 smithi153 bash[14340]: audit 2024-03-12T03:07:06.403279+0000 mgr.smithi153.asomxr (mgr.14195) 2102 : audit 0 from='client.15694 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:07.253 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:07:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:06 smithi163 bash[22347]: cluster 2024-03-12T03:07:05.925950+0000 mgr.smithi153.asomxr (mgr.14195) 2101 : cluster 0 pgmap v1755: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:07.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:06 smithi163 bash[22347]: audit 2024-03-12T03:07:06.403279+0000 mgr.smithi153.asomxr (mgr.14195) 2102 : audit 0 from='client.15694 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:08.258 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:07:09.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:08 smithi153 bash[14340]: cluster 2024-03-12T03:07:07.926432+0000 mgr.smithi153.asomxr (mgr.14195) 2103 : cluster 0 pgmap v1756: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:09.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:08 smithi163 bash[22347]: cluster 2024-03-12T03:07:07.926432+0000 mgr.smithi153.asomxr (mgr.14195) 2103 : cluster 0 pgmap v1756: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:11.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:10 smithi153 bash[14340]: cluster 2024-03-12T03:07:09.927076+0000 mgr.smithi153.asomxr (mgr.14195) 2104 : cluster 0 pgmap v1757: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:11.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:10 smithi163 bash[22347]: cluster 2024-03-12T03:07:09.927076+0000 mgr.smithi153.asomxr (mgr.14195) 2104 : cluster 0 pgmap v1757: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:13.013 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:07:13.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:12 smithi153 bash[14340]: cluster 2024-03-12T03:07:11.927700+0000 mgr.smithi153.asomxr (mgr.14195) 2105 : cluster 0 pgmap v1758: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:13.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:12 smithi163 bash[22347]: cluster 2024-03-12T03:07:11.927700+0000 mgr.smithi153.asomxr (mgr.14195) 2105 : cluster 0 pgmap v1758: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:14.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:14 smithi153 bash[14340]: cluster 2024-03-12T03:07:13.928085+0000 mgr.smithi153.asomxr (mgr.14195) 2106 : cluster 0 pgmap v1759: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:14.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:14 smithi163 bash[22347]: cluster 2024-03-12T03:07:13.928085+0000 mgr.smithi153.asomxr (mgr.14195) 2106 : cluster 0 pgmap v1759: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:15.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:15 smithi153 bash[14340]: audit 2024-03-12T03:07:15.466980+0000 mon.smithi153 (mon.0) 1318 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:07:15.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:15 smithi163 bash[22347]: audit 2024-03-12T03:07:15.466980+0000 mon.smithi153 (mon.0) 1318 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:07:16.390 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:07:16.390 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:07:16.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:16 smithi163 bash[22347]: cluster 2024-03-12T03:07:15.928524+0000 mgr.smithi153.asomxr (mgr.14195) 2107 : cluster 0 pgmap v1760: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:16.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:16 smithi163 bash[22347]: audit 2024-03-12T03:07:16.373778+0000 mgr.smithi153.asomxr (mgr.14195) 2108 : audit 0 from='client.15698 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:17.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:16 smithi153 bash[14340]: cluster 2024-03-12T03:07:15.928524+0000 mgr.smithi153.asomxr (mgr.14195) 2107 : cluster 0 pgmap v1760: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:17.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:16 smithi153 bash[14340]: audit 2024-03-12T03:07:16.373778+0000 mgr.smithi153.asomxr (mgr.14195) 2108 : audit 0 from='client.15698 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:17.180 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:07:18.181 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:07:19.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:18 smithi153 bash[14340]: cluster 2024-03-12T03:07:17.929102+0000 mgr.smithi153.asomxr (mgr.14195) 2109 : cluster 0 pgmap v1761: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:19.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:18 smithi163 bash[22347]: cluster 2024-03-12T03:07:17.929102+0000 mgr.smithi153.asomxr (mgr.14195) 2109 : cluster 0 pgmap v1761: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:21.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:20 smithi153 bash[14340]: cluster 2024-03-12T03:07:19.929751+0000 mgr.smithi153.asomxr (mgr.14195) 2110 : cluster 0 pgmap v1762: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:21.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:20 smithi153 bash[14340]: audit 2024-03-12T03:07:20.014475+0000 mon.smithi153 (mon.0) 1319 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:07:21.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:20 smithi153 bash[14340]: audit 2024-03-12T03:07:20.594382+0000 mon.smithi153 (mon.0) 1320 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:07:21.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:20 smithi153 bash[14340]: audit 2024-03-12T03:07:20.595808+0000 mon.smithi153 (mon.0) 1321 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:07:21.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:20 smithi153 bash[14340]: audit 2024-03-12T03:07:20.604461+0000 mon.smithi153 (mon.0) 1322 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:07:21.323 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:20 smithi153 bash[14340]: audit 2024-03-12T03:07:20.610336+0000 mon.smithi153 (mon.0) 1323 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:07:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:20 smithi163 bash[22347]: cluster 2024-03-12T03:07:19.929751+0000 mgr.smithi153.asomxr (mgr.14195) 2110 : cluster 0 pgmap v1762: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:20 smithi163 bash[22347]: audit 2024-03-12T03:07:20.014475+0000 mon.smithi153 (mon.0) 1319 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:07:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:20 smithi163 bash[22347]: audit 2024-03-12T03:07:20.594382+0000 mon.smithi153 (mon.0) 1320 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:07:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:20 smithi163 bash[22347]: audit 2024-03-12T03:07:20.595808+0000 mon.smithi153 (mon.0) 1321 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:07:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:20 smithi163 bash[22347]: audit 2024-03-12T03:07:20.604461+0000 mon.smithi153 (mon.0) 1322 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:07:21.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:20 smithi163 bash[22347]: audit 2024-03-12T03:07:20.610336+0000 mon.smithi153 (mon.0) 1323 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:07:22.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:22 smithi163 bash[22347]: cluster 2024-03-12T03:07:20.598003+0000 mgr.smithi153.asomxr (mgr.14195) 2111 : cluster 0 pgmap v1763: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:22.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:22 smithi163 bash[22347]: cluster 2024-03-12T03:07:20.598314+0000 mgr.smithi153.asomxr (mgr.14195) 2112 : cluster 0 pgmap v1764: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:22.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:21 smithi153 bash[14340]: cluster 2024-03-12T03:07:20.598003+0000 mgr.smithi153.asomxr (mgr.14195) 2111 : cluster 0 pgmap v1763: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:22.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:21 smithi153 bash[14340]: cluster 2024-03-12T03:07:20.598314+0000 mgr.smithi153.asomxr (mgr.14195) 2112 : cluster 0 pgmap v1764: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:22.923 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:07:24.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:24 smithi153 bash[14340]: cluster 2024-03-12T03:07:22.598887+0000 mgr.smithi153.asomxr (mgr.14195) 2113 : cluster 0 pgmap v1765: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:24.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:24 smithi163 bash[22347]: cluster 2024-03-12T03:07:22.598887+0000 mgr.smithi153.asomxr (mgr.14195) 2113 : cluster 0 pgmap v1765: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:26.294 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:07:26.294 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:07:26.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:26 smithi153 bash[14340]: cluster 2024-03-12T03:07:24.599290+0000 mgr.smithi153.asomxr (mgr.14195) 2114 : cluster 0 pgmap v1766: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:26.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:26 smithi163 bash[22347]: cluster 2024-03-12T03:07:24.599290+0000 mgr.smithi153.asomxr (mgr.14195) 2114 : cluster 0 pgmap v1766: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:27.111 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:07:27.233 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:27 smithi163 bash[22347]: audit 2024-03-12T03:07:26.278523+0000 mgr.smithi153.asomxr (mgr.14195) 2115 : audit 0 from='client.15702 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:27.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:27 smithi153 bash[14340]: audit 2024-03-12T03:07:26.278523+0000 mgr.smithi153.asomxr (mgr.14195) 2115 : audit 0 from='client.15702 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:28.112 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:07:28.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:28 smithi153 bash[14340]: cluster 2024-03-12T03:07:26.599881+0000 mgr.smithi153.asomxr (mgr.14195) 2116 : cluster 0 pgmap v1767: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:28 smithi163 bash[22347]: cluster 2024-03-12T03:07:26.599881+0000 mgr.smithi153.asomxr (mgr.14195) 2116 : cluster 0 pgmap v1767: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:29.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:29 smithi163 bash[22347]: cluster 2024-03-12T03:07:28.600515+0000 mgr.smithi153.asomxr (mgr.14195) 2117 : cluster 0 pgmap v1768: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:30.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:29 smithi153 bash[14340]: cluster 2024-03-12T03:07:28.600515+0000 mgr.smithi153.asomxr (mgr.14195) 2117 : cluster 0 pgmap v1768: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:30.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:30 smithi163 bash[22347]: audit 2024-03-12T03:07:30.467212+0000 mon.smithi153 (mon.0) 1324 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:07:31.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:30 smithi153 bash[14340]: audit 2024-03-12T03:07:30.467212+0000 mon.smithi153 (mon.0) 1324 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:07:31.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:31 smithi153 bash[14340]: cluster 2024-03-12T03:07:30.601237+0000 mgr.smithi153.asomxr (mgr.14195) 2118 : cluster 0 pgmap v1769: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:31.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:31 smithi163 bash[22347]: cluster 2024-03-12T03:07:30.601237+0000 mgr.smithi153.asomxr (mgr.14195) 2118 : cluster 0 pgmap v1769: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:32.862 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:07:33.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:33 smithi163 bash[22347]: cluster 2024-03-12T03:07:32.601914+0000 mgr.smithi153.asomxr (mgr.14195) 2119 : cluster 0 pgmap v1770: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:34.036 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:33 smithi153 bash[14340]: cluster 2024-03-12T03:07:32.601914+0000 mgr.smithi153.asomxr (mgr.14195) 2119 : cluster 0 pgmap v1770: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:35.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:35 smithi153 bash[14340]: cluster 2024-03-12T03:07:34.602681+0000 mgr.smithi153.asomxr (mgr.14195) 2120 : cluster 0 pgmap v1771: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:35.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:35 smithi163 bash[22347]: cluster 2024-03-12T03:07:34.602681+0000 mgr.smithi153.asomxr (mgr.14195) 2120 : cluster 0 pgmap v1771: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:36.315 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:07:36.315 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:07:36.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:36 smithi153 bash[14340]: audit 2024-03-12T03:07:36.305012+0000 mgr.smithi153.asomxr (mgr.14195) 2121 : audit 0 from='client.15706 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:36.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:36 smithi163 bash[22347]: audit 2024-03-12T03:07:36.305012+0000 mgr.smithi153.asomxr (mgr.14195) 2121 : audit 0 from='client.15706 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:37.303 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:07:37.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:37 smithi163 bash[22347]: cluster 2024-03-12T03:07:36.603073+0000 mgr.smithi153.asomxr (mgr.14195) 2122 : cluster 0 pgmap v1772: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:38.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:37 smithi153 bash[14340]: cluster 2024-03-12T03:07:36.603073+0000 mgr.smithi153.asomxr (mgr.14195) 2122 : cluster 0 pgmap v1772: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:38.304 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:07:39.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:39 smithi163 bash[22347]: cluster 2024-03-12T03:07:38.603637+0000 mgr.smithi153.asomxr (mgr.14195) 2123 : cluster 0 pgmap v1773: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:40.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:39 smithi153 bash[14340]: cluster 2024-03-12T03:07:38.603637+0000 mgr.smithi153.asomxr (mgr.14195) 2123 : cluster 0 pgmap v1773: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:41.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:41 smithi163 bash[22347]: cluster 2024-03-12T03:07:40.604242+0000 mgr.smithi153.asomxr (mgr.14195) 2124 : cluster 0 pgmap v1774: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:42.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:41 smithi153 bash[14340]: cluster 2024-03-12T03:07:40.604242+0000 mgr.smithi153.asomxr (mgr.14195) 2124 : cluster 0 pgmap v1774: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:43.054 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:07:43.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:43 smithi163 bash[22347]: cluster 2024-03-12T03:07:42.604800+0000 mgr.smithi153.asomxr (mgr.14195) 2125 : cluster 0 pgmap v1775: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:44.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:43 smithi153 bash[14340]: cluster 2024-03-12T03:07:42.604800+0000 mgr.smithi153.asomxr (mgr.14195) 2125 : cluster 0 pgmap v1775: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:45.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:45 smithi163 bash[22347]: cluster 2024-03-12T03:07:44.605474+0000 mgr.smithi153.asomxr (mgr.14195) 2126 : cluster 0 pgmap v1776: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:45.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:45 smithi163 bash[22347]: audit 2024-03-12T03:07:45.467609+0000 mon.smithi153 (mon.0) 1325 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:07:46.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:45 smithi153 bash[14340]: cluster 2024-03-12T03:07:44.605474+0000 mgr.smithi153.asomxr (mgr.14195) 2126 : cluster 0 pgmap v1776: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:46.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:45 smithi153 bash[14340]: audit 2024-03-12T03:07:45.467609+0000 mon.smithi153 (mon.0) 1325 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:07:46.276 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:07:46.276 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:07:46.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:46 smithi153 bash[14340]: audit 2024-03-12T03:07:46.262547+0000 mgr.smithi153.asomxr (mgr.14195) 2127 : audit 0 from='client.15710 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:46.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:46 smithi163 bash[22347]: audit 2024-03-12T03:07:46.262547+0000 mgr.smithi153.asomxr (mgr.14195) 2127 : audit 0 from='client.15710 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:47.211 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:07:47.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:47 smithi163 bash[22347]: cluster 2024-03-12T03:07:46.605997+0000 mgr.smithi153.asomxr (mgr.14195) 2128 : cluster 0 pgmap v1777: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:48.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:47 smithi153 bash[14340]: cluster 2024-03-12T03:07:46.605997+0000 mgr.smithi153.asomxr (mgr.14195) 2128 : cluster 0 pgmap v1777: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:48.212 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:07:49.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:49 smithi163 bash[22347]: cluster 2024-03-12T03:07:48.606562+0000 mgr.smithi153.asomxr (mgr.14195) 2129 : cluster 0 pgmap v1778: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:50.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:49 smithi153 bash[14340]: cluster 2024-03-12T03:07:48.606562+0000 mgr.smithi153.asomxr (mgr.14195) 2129 : cluster 0 pgmap v1778: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:51.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:51 smithi163 bash[22347]: cluster 2024-03-12T03:07:50.607199+0000 mgr.smithi153.asomxr (mgr.14195) 2130 : cluster 0 pgmap v1779: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:52.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:51 smithi153 bash[14340]: cluster 2024-03-12T03:07:50.607199+0000 mgr.smithi153.asomxr (mgr.14195) 2130 : cluster 0 pgmap v1779: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:52.964 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:07:53.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:53 smithi163 bash[22347]: cluster 2024-03-12T03:07:52.607881+0000 mgr.smithi153.asomxr (mgr.14195) 2131 : cluster 0 pgmap v1780: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:54.025 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:53 smithi153 bash[14340]: cluster 2024-03-12T03:07:52.607881+0000 mgr.smithi153.asomxr (mgr.14195) 2131 : cluster 0 pgmap v1780: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:55.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:55 smithi163 bash[22347]: cluster 2024-03-12T03:07:54.608595+0000 mgr.smithi153.asomxr (mgr.14195) 2132 : cluster 0 pgmap v1781: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:56.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:55 smithi153 bash[14340]: cluster 2024-03-12T03:07:54.608595+0000 mgr.smithi153.asomxr (mgr.14195) 2132 : cluster 0 pgmap v1781: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:56.291 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:07:56.292 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:07:56.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:56 smithi153 bash[14340]: audit 2024-03-12T03:07:56.277456+0000 mgr.smithi153.asomxr (mgr.14195) 2133 : audit 0 from='client.15714 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:56.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:56 smithi163 bash[22347]: audit 2024-03-12T03:07:56.277456+0000 mgr.smithi153.asomxr (mgr.14195) 2133 : audit 0 from='client.15714 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:07:57.127 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:07:57.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:57 smithi163 bash[22347]: cluster 2024-03-12T03:07:56.609101+0000 mgr.smithi153.asomxr (mgr.14195) 2134 : cluster 0 pgmap v1782: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:58.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:57 smithi153 bash[14340]: cluster 2024-03-12T03:07:56.609101+0000 mgr.smithi153.asomxr (mgr.14195) 2134 : cluster 0 pgmap v1782: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:07:58.128 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:07:59.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:07:59 smithi163 bash[22347]: cluster 2024-03-12T03:07:58.609626+0000 mgr.smithi153.asomxr (mgr.14195) 2135 : cluster 0 pgmap v1783: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:00.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:07:59 smithi153 bash[14340]: cluster 2024-03-12T03:07:58.609626+0000 mgr.smithi153.asomxr (mgr.14195) 2135 : cluster 0 pgmap v1783: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:00.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:00 smithi163 bash[22347]: audit 2024-03-12T03:08:00.468146+0000 mon.smithi153 (mon.0) 1326 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:08:01.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:00 smithi153 bash[14340]: audit 2024-03-12T03:08:00.468146+0000 mon.smithi153 (mon.0) 1326 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:08:01.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:01 smithi163 bash[22347]: cluster 2024-03-12T03:08:00.610246+0000 mgr.smithi153.asomxr (mgr.14195) 2136 : cluster 0 pgmap v1784: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:02.022 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:01 smithi153 bash[14340]: cluster 2024-03-12T03:08:00.610246+0000 mgr.smithi153.asomxr (mgr.14195) 2136 : cluster 0 pgmap v1784: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:08:03.957 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:03 smithi153 bash[14340]: cluster 2024-03-12T03:08:02.610938+0000 mgr.smithi153.asomxr (mgr.14195) 2137 : cluster 0 pgmap v1785: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:03.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:03 smithi163 bash[22347]: cluster 2024-03-12T03:08:02.610938+0000 mgr.smithi153.asomxr (mgr.14195) 2137 : cluster 0 pgmap v1785: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:05.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:05 smithi153 bash[14340]: cluster 2024-03-12T03:08:04.611626+0000 mgr.smithi153.asomxr (mgr.14195) 2138 : cluster 0 pgmap v1786: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:05.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:05 smithi163 bash[22347]: cluster 2024-03-12T03:08:04.611626+0000 mgr.smithi153.asomxr (mgr.14195) 2138 : cluster 0 pgmap v1786: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:06.303 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:08:06.303 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:08:06.892 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:06 smithi153 bash[14340]: audit 2024-03-12T03:08:06.288547+0000 mgr.smithi153.asomxr (mgr.14195) 2139 : audit 0 from='client.15718 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:06.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:06 smithi163 bash[22347]: audit 2024-03-12T03:08:06.288547+0000 mgr.smithi153.asomxr (mgr.14195) 2139 : audit 0 from='client.15718 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:07.163 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:08:07.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:07 smithi163 bash[22347]: cluster 2024-03-12T03:08:06.612292+0000 mgr.smithi153.asomxr (mgr.14195) 2140 : cluster 0 pgmap v1787: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:08.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:07 smithi153 bash[14340]: cluster 2024-03-12T03:08:06.612292+0000 mgr.smithi153.asomxr (mgr.14195) 2140 : cluster 0 pgmap v1787: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:08.164 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:08:09.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:09 smithi163 bash[22347]: cluster 2024-03-12T03:08:08.612966+0000 mgr.smithi153.asomxr (mgr.14195) 2141 : cluster 0 pgmap v1788: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:10.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:09 smithi153 bash[14340]: cluster 2024-03-12T03:08:08.612966+0000 mgr.smithi153.asomxr (mgr.14195) 2141 : cluster 0 pgmap v1788: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:11.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:11 smithi163 bash[22347]: cluster 2024-03-12T03:08:10.613630+0000 mgr.smithi153.asomxr (mgr.14195) 2142 : cluster 0 pgmap v1789: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:12.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:11 smithi153 bash[14340]: cluster 2024-03-12T03:08:10.613630+0000 mgr.smithi153.asomxr (mgr.14195) 2142 : cluster 0 pgmap v1789: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:12.914 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:08:13.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:13 smithi163 bash[22347]: cluster 2024-03-12T03:08:12.614201+0000 mgr.smithi153.asomxr (mgr.14195) 2143 : cluster 0 pgmap v1790: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:14.005 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:13 smithi153 bash[14340]: cluster 2024-03-12T03:08:12.614201+0000 mgr.smithi153.asomxr (mgr.14195) 2143 : cluster 0 pgmap v1790: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:15.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:15 smithi163 bash[22347]: cluster 2024-03-12T03:08:14.614946+0000 mgr.smithi153.asomxr (mgr.14195) 2144 : cluster 0 pgmap v1791: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:15.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:15 smithi163 bash[22347]: audit 2024-03-12T03:08:15.468431+0000 mon.smithi153 (mon.0) 1327 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:08:16.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:15 smithi153 bash[14340]: cluster 2024-03-12T03:08:14.614946+0000 mgr.smithi153.asomxr (mgr.14195) 2144 : cluster 0 pgmap v1791: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:16.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:15 smithi153 bash[14340]: audit 2024-03-12T03:08:15.468431+0000 mon.smithi153 (mon.0) 1327 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:08:16.233 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:08:16.233 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:08:16.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:16 smithi163 bash[22347]: audit 2024-03-12T03:08:16.218629+0000 mgr.smithi153.asomxr (mgr.14195) 2145 : audit 0 from='client.15722 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:17.037 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:16 smithi153 bash[14340]: audit 2024-03-12T03:08:16.218629+0000 mgr.smithi153.asomxr (mgr.14195) 2145 : audit 0 from='client.15722 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:17.038 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:08:17.977 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:17 smithi163 bash[22347]: cluster 2024-03-12T03:08:16.615415+0000 mgr.smithi153.asomxr (mgr.14195) 2146 : cluster 0 pgmap v1792: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:18.039 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:08:18.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:17 smithi153 bash[14340]: cluster 2024-03-12T03:08:16.615415+0000 mgr.smithi153.asomxr (mgr.14195) 2146 : cluster 0 pgmap v1792: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:19.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:19 smithi163 bash[22347]: cluster 2024-03-12T03:08:18.615911+0000 mgr.smithi153.asomxr (mgr.14195) 2147 : cluster 0 pgmap v1793: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:20.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:19 smithi153 bash[14340]: cluster 2024-03-12T03:08:18.615911+0000 mgr.smithi153.asomxr (mgr.14195) 2147 : cluster 0 pgmap v1793: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:21.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:21 smithi163 bash[22347]: cluster 2024-03-12T03:08:20.616564+0000 mgr.smithi153.asomxr (mgr.14195) 2148 : cluster 0 pgmap v1794: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:21.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:21 smithi163 bash[22347]: audit 2024-03-12T03:08:20.701858+0000 mon.smithi153 (mon.0) 1328 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:08:21.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:21 smithi163 bash[22347]: audit 2024-03-12T03:08:21.281526+0000 mon.smithi153 (mon.0) 1329 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:08:21.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:21 smithi163 bash[22347]: audit 2024-03-12T03:08:21.283218+0000 mon.smithi153 (mon.0) 1330 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:08:21.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:21 smithi163 bash[22347]: cluster 2024-03-12T03:08:21.285928+0000 mgr.smithi153.asomxr (mgr.14195) 2149 : cluster 0 pgmap v1795: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:21.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:21 smithi163 bash[22347]: cluster 2024-03-12T03:08:21.286335+0000 mgr.smithi153.asomxr (mgr.14195) 2150 : cluster 0 pgmap v1796: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:21.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:21 smithi163 bash[22347]: audit 2024-03-12T03:08:21.292388+0000 mon.smithi153 (mon.0) 1331 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:08:21.984 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:21 smithi163 bash[22347]: audit 2024-03-12T03:08:21.298151+0000 mon.smithi153 (mon.0) 1332 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:08:22.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:21 smithi153 bash[14340]: cluster 2024-03-12T03:08:20.616564+0000 mgr.smithi153.asomxr (mgr.14195) 2148 : cluster 0 pgmap v1794: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:22.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:21 smithi153 bash[14340]: audit 2024-03-12T03:08:20.701858+0000 mon.smithi153 (mon.0) 1328 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-12T03:08:22.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:21 smithi153 bash[14340]: audit 2024-03-12T03:08:21.281526+0000 mon.smithi153 (mon.0) 1329 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-12T03:08:22.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:21 smithi153 bash[14340]: audit 2024-03-12T03:08:21.283218+0000 mon.smithi153 (mon.0) 1330 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-12T03:08:22.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:21 smithi153 bash[14340]: cluster 2024-03-12T03:08:21.285928+0000 mgr.smithi153.asomxr (mgr.14195) 2149 : cluster 0 pgmap v1795: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:22.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:21 smithi153 bash[14340]: cluster 2024-03-12T03:08:21.286335+0000 mgr.smithi153.asomxr (mgr.14195) 2150 : cluster 0 pgmap v1796: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:22.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:21 smithi153 bash[14340]: audit 2024-03-12T03:08:21.292388+0000 mon.smithi153 (mon.0) 1331 : audit 1 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' 2024-03-12T03:08:22.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:21 smithi153 bash[14340]: audit 2024-03-12T03:08:21.298151+0000 mon.smithi153 (mon.0) 1332 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-12T03:08:22.793 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:08:24.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:24 smithi153 bash[14340]: cluster 2024-03-12T03:08:23.287048+0000 mgr.smithi153.asomxr (mgr.14195) 2151 : cluster 0 pgmap v1797: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:24.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:24 smithi163 bash[22347]: cluster 2024-03-12T03:08:23.287048+0000 mgr.smithi153.asomxr (mgr.14195) 2151 : cluster 0 pgmap v1797: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:26.224 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:08:26.224 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:08:26.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:26 smithi153 bash[14340]: cluster 2024-03-12T03:08:25.287752+0000 mgr.smithi153.asomxr (mgr.14195) 2152 : cluster 0 pgmap v1798: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:26.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:26 smithi163 bash[22347]: cluster 2024-03-12T03:08:25.287752+0000 mgr.smithi153.asomxr (mgr.14195) 2152 : cluster 0 pgmap v1798: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:27.030 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:08:27.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:27 smithi163 bash[22347]: audit 2024-03-12T03:08:26.210218+0000 mgr.smithi153.asomxr (mgr.14195) 2153 : audit 0 from='client.15726 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:27.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:27 smithi153 bash[14340]: audit 2024-03-12T03:08:26.210218+0000 mgr.smithi153.asomxr (mgr.14195) 2153 : audit 0 from='client.15726 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:28.031 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:08:28.483 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:28 smithi163 bash[22347]: cluster 2024-03-12T03:08:27.288377+0000 mgr.smithi153.asomxr (mgr.14195) 2154 : cluster 0 pgmap v1799: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:28.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:28 smithi153 bash[14340]: cluster 2024-03-12T03:08:27.288377+0000 mgr.smithi153.asomxr (mgr.14195) 2154 : cluster 0 pgmap v1799: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:29.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:29 smithi163 bash[22347]: cluster 2024-03-12T03:08:29.289046+0000 mgr.smithi153.asomxr (mgr.14195) 2155 : cluster 0 pgmap v1800: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:30.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:29 smithi153 bash[14340]: cluster 2024-03-12T03:08:29.289046+0000 mgr.smithi153.asomxr (mgr.14195) 2155 : cluster 0 pgmap v1800: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:30.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:30 smithi163 bash[22347]: audit 2024-03-12T03:08:30.469046+0000 mon.smithi153 (mon.0) 1333 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:08:31.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:30 smithi153 bash[14340]: audit 2024-03-12T03:08:30.469046+0000 mon.smithi153 (mon.0) 1333 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:08:31.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:31 smithi163 bash[22347]: cluster 2024-03-12T03:08:31.289675+0000 mgr.smithi153.asomxr (mgr.14195) 2156 : cluster 0 pgmap v1801: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:32.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:31 smithi153 bash[14340]: cluster 2024-03-12T03:08:31.289675+0000 mgr.smithi153.asomxr (mgr.14195) 2156 : cluster 0 pgmap v1801: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:32.790 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:08:34.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:34 smithi153 bash[14340]: cluster 2024-03-12T03:08:33.290264+0000 mgr.smithi153.asomxr (mgr.14195) 2157 : cluster 0 pgmap v1802: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:34.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:34 smithi163 bash[22347]: cluster 2024-03-12T03:08:33.290264+0000 mgr.smithi153.asomxr (mgr.14195) 2157 : cluster 0 pgmap v1802: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:36.124 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:08:36.125 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:08:36.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:36 smithi153 bash[14340]: cluster 2024-03-12T03:08:35.291020+0000 mgr.smithi153.asomxr (mgr.14195) 2158 : cluster 0 pgmap v1803: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:36.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:36 smithi163 bash[22347]: cluster 2024-03-12T03:08:35.291020+0000 mgr.smithi153.asomxr (mgr.14195) 2158 : cluster 0 pgmap v1803: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:36.906 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:08:37.729 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:37 smithi153 bash[14340]: audit 2024-03-12T03:08:36.115283+0000 mgr.smithi153.asomxr (mgr.14195) 2159 : audit 0 from='client.15730 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:37.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:37 smithi163 bash[22347]: audit 2024-03-12T03:08:36.115283+0000 mgr.smithi153.asomxr (mgr.14195) 2159 : audit 0 from='client.15730 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:37.908 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:08:38.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:38 smithi163 bash[22347]: cluster 2024-03-12T03:08:37.291652+0000 mgr.smithi153.asomxr (mgr.14195) 2160 : cluster 0 pgmap v1804: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:38.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:38 smithi153 bash[14340]: cluster 2024-03-12T03:08:37.291652+0000 mgr.smithi153.asomxr (mgr.14195) 2160 : cluster 0 pgmap v1804: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:39.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:39 smithi163 bash[22347]: cluster 2024-03-12T03:08:39.292347+0000 mgr.smithi153.asomxr (mgr.14195) 2161 : cluster 0 pgmap v1805: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:40.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:39 smithi153 bash[14340]: cluster 2024-03-12T03:08:39.292347+0000 mgr.smithi153.asomxr (mgr.14195) 2161 : cluster 0 pgmap v1805: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:42.666 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:08:42.700 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:42 smithi153 bash[14340]: cluster 2024-03-12T03:08:41.293043+0000 mgr.smithi153.asomxr (mgr.14195) 2162 : cluster 0 pgmap v1806: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:42.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:42 smithi163 bash[22347]: cluster 2024-03-12T03:08:41.293043+0000 mgr.smithi153.asomxr (mgr.14195) 2162 : cluster 0 pgmap v1806: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:44.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:44 smithi153 bash[14340]: cluster 2024-03-12T03:08:43.293654+0000 mgr.smithi153.asomxr (mgr.14195) 2163 : cluster 0 pgmap v1807: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:44.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:44 smithi163 bash[22347]: cluster 2024-03-12T03:08:43.293654+0000 mgr.smithi153.asomxr (mgr.14195) 2163 : cluster 0 pgmap v1807: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:46.149 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:08:46.149 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:08:46.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:46 smithi153 bash[14340]: cluster 2024-03-12T03:08:45.294188+0000 mgr.smithi153.asomxr (mgr.14195) 2164 : cluster 0 pgmap v1808: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:46.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:46 smithi153 bash[14340]: audit 2024-03-12T03:08:45.469389+0000 mon.smithi153 (mon.0) 1334 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:08:46.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:46 smithi163 bash[22347]: cluster 2024-03-12T03:08:45.294188+0000 mgr.smithi153.asomxr (mgr.14195) 2164 : cluster 0 pgmap v1808: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:46.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:46 smithi163 bash[22347]: audit 2024-03-12T03:08:45.469389+0000 mon.smithi153 (mon.0) 1334 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:08:47.001 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:08:47.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:47 smithi163 bash[22347]: audit 2024-03-12T03:08:46.139188+0000 mgr.smithi153.asomxr (mgr.14195) 2165 : audit 0 from='client.15734 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:47.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:47 smithi153 bash[14340]: audit 2024-03-12T03:08:46.139188+0000 mgr.smithi153.asomxr (mgr.14195) 2165 : audit 0 from='client.15734 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:48.003 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:08:48.629 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:48 smithi163 bash[22347]: cluster 2024-03-12T03:08:47.294675+0000 mgr.smithi153.asomxr (mgr.14195) 2166 : cluster 0 pgmap v1809: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:48.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:48 smithi153 bash[14340]: cluster 2024-03-12T03:08:47.294675+0000 mgr.smithi153.asomxr (mgr.14195) 2166 : cluster 0 pgmap v1809: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:49.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:49 smithi163 bash[22347]: cluster 2024-03-12T03:08:49.295339+0000 mgr.smithi153.asomxr (mgr.14195) 2167 : cluster 0 pgmap v1810: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:50.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:49 smithi153 bash[14340]: cluster 2024-03-12T03:08:49.295339+0000 mgr.smithi153.asomxr (mgr.14195) 2167 : cluster 0 pgmap v1810: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:52.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:52 smithi163 bash[22347]: cluster 2024-03-12T03:08:51.295976+0000 mgr.smithi153.asomxr (mgr.14195) 2168 : cluster 0 pgmap v1811: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:52.744 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:08:52.778 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:52 smithi153 bash[14340]: cluster 2024-03-12T03:08:51.295976+0000 mgr.smithi153.asomxr (mgr.14195) 2168 : cluster 0 pgmap v1811: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:54.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:54 smithi153 bash[14340]: cluster 2024-03-12T03:08:53.296585+0000 mgr.smithi153.asomxr (mgr.14195) 2169 : cluster 0 pgmap v1812: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:54.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:54 smithi163 bash[22347]: cluster 2024-03-12T03:08:53.296585+0000 mgr.smithi153.asomxr (mgr.14195) 2169 : cluster 0 pgmap v1812: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:56.014 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:08:56.015 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:08:56.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:56 smithi153 bash[14340]: cluster 2024-03-12T03:08:55.297039+0000 mgr.smithi153.asomxr (mgr.14195) 2170 : cluster 0 pgmap v1813: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:56.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:56 smithi163 bash[22347]: cluster 2024-03-12T03:08:55.297039+0000 mgr.smithi153.asomxr (mgr.14195) 2170 : cluster 0 pgmap v1813: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:56.800 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:08:57.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:57 smithi163 bash[22347]: audit 2024-03-12T03:08:56.001163+0000 mgr.smithi153.asomxr (mgr.14195) 2171 : audit 0 from='client.15738 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:57.801 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c2ea977b6956d1f694ccf66acc692fdd979e3baa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 -- ceph orch ls -f json 2024-03-12T03:08:57.815 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:57 smithi153 bash[14340]: audit 2024-03-12T03:08:56.001163+0000 mgr.smithi153.asomxr (mgr.14195) 2171 : audit 0 from='client.15738 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-12T03:08:58.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:58 smithi163 bash[22347]: cluster 2024-03-12T03:08:57.297559+0000 mgr.smithi153.asomxr (mgr.14195) 2172 : cluster 0 pgmap v1814: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:58.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:58 smithi153 bash[14340]: cluster 2024-03-12T03:08:57.297559+0000 mgr.smithi153.asomxr (mgr.14195) 2172 : cluster 0 pgmap v1814: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:08:59.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:08:59 smithi163 bash[22347]: cluster 2024-03-12T03:08:59.298234+0000 mgr.smithi153.asomxr (mgr.14195) 2173 : cluster 0 pgmap v1815: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:09:00.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:08:59 smithi153 bash[14340]: cluster 2024-03-12T03:08:59.298234+0000 mgr.smithi153.asomxr (mgr.14195) 2173 : cluster 0 pgmap v1815: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:09:00.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:09:00 smithi163 bash[22347]: audit 2024-03-12T03:09:00.469854+0000 mon.smithi153 (mon.0) 1335 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:09:01.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:09:00 smithi153 bash[14340]: audit 2024-03-12T03:09:00.469854+0000 mon.smithi153 (mon.0) 1335 : audit 0 from='mgr.14195 172.21.15.153:0/1650549236' entity='mgr.smithi153.asomxr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-12T03:09:01.983 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:09:01 smithi163 bash[22347]: cluster 2024-03-12T03:09:01.298886+0000 mgr.smithi153.asomxr (mgr.14195) 2174 : cluster 0 pgmap v1816: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:09:02.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:09:01 smithi153 bash[14340]: cluster 2024-03-12T03:09:01.298886+0000 mgr.smithi153.asomxr (mgr.14195) 2174 : cluster 0 pgmap v1816: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:09:02.559 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/mon.smithi153/config 2024-03-12T03:09:04.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:09:04 smithi163 bash[22347]: cluster 2024-03-12T03:09:03.299465+0000 mgr.smithi153.asomxr (mgr.14195) 2175 : cluster 0 pgmap v1817: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:09:04.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:09:04 smithi153 bash[14340]: cluster 2024-03-12T03:09:03.299465+0000 mgr.smithi153.asomxr (mgr.14195) 2175 : cluster 0 pgmap v1817: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:09:05.975 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-12T03:09:05.975 INFO:teuthology.orchestra.run.smithi153.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2024-03-12T02:10:13.890578Z", "last_refresh": "2024-03-12T03:02:10.434337Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:15.514013Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2024-03-12T02:10:04.775605Z", "last_refresh": "2024-03-12T03:02:10.433664Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:18.359636Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2024-03-12T02:10:02.575298Z", "last_refresh": "2024-03-12T03:02:10.433900Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:17:51.355110Z service:elasticsearch [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "elasticsearch", "service_type": "elasticsearch", "status": {"created": "2024-03-12T02:17:51.348737Z", "last_refresh": "2024-03-12T03:04:18.043013Z", "ports": [9200], "running": 1, "size": 1}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2024-03-12T02:10:08.966951Z", "last_refresh": "2024-03-12T03:02:10.434549Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.369561Z service:jaeger-agent [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "jaeger-agent", "service_type": "jaeger-agent", "status": {"created": "2024-03-12T02:17:51.365615Z", "last_refresh": "2024-03-12T03:02:10.435727Z", "ports": [6799], "running": 0, "size": 2}}, {"events": ["2024-03-12T02:17:51.360419Z service:jaeger-collector [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-collector", "service_type": "jaeger-collector", "status": {"created": "2024-03-12T02:17:51.355663Z", "last_refresh": "2024-03-12T03:02:10.435937Z", "ports": [14250], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:17:51.365068Z service:jaeger-query [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "jaeger-query", "service_type": "jaeger-query", "status": {"created": "2024-03-12T02:17:51.360910Z", "last_refresh": "2024-03-12T03:04:18.043248Z", "ports": [16686], "running": 1, "size": 1}}, {"events": ["2024-03-12T02:12:22.991140Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2024-03-12T02:10:00.490125Z", "last_refresh": "2024-03-12T03:02:10.433375Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:01.314497Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["smithi153:172.21.15.153=smithi153", "smithi163:172.21.15.163=smithi163"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2024-03-12T02:12:01.311559Z", "last_refresh": "2024-03-12T03:02:10.433073Z", "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:19.440958Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2024-03-12T02:10:11.558888Z", "last_refresh": "2024-03-12T03:02:10.434128Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2024-03-12T02:12:57.777057Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2024-03-12T02:12:57.770579Z", "last_refresh": "2024-03-12T03:02:10.434970Z", "running": 8, "size": 8}}, {"events": ["2024-03-12T02:12:29.350565Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2024-03-12T02:10:06.904192Z", "last_refresh": "2024-03-12T03:02:10.434759Z", "ports": [9095], "running": 1, "size": 1}}] 2024-03-12T03:09:06.572 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:09:06 smithi153 bash[14340]: cluster 2024-03-12T03:09:05.300190+0000 mgr.smithi153.asomxr (mgr.14195) 2176 : cluster 0 pgmap v1818: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:09:06.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:09:06 smithi163 bash[22347]: cluster 2024-03-12T03:09:05.300190+0000 mgr.smithi153.asomxr (mgr.14195) 2176 : cluster 0 pgmap v1818: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-12T03:09:06.806 INFO:tasks.cephadm:jaeger-agent has 0/2 2024-03-12T03:09:06.806 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_583c7cd94fcfcfa9abcbbb46bafd029ba8504cb0/qa/tasks/cephadm.py", line 1368, in wait_for_service while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (301) after waiting for 300 seconds 2024-03-12T03:09:07.027 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=dfc02bace98345b2b0255bc249418893 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_583c7cd94fcfcfa9abcbbb46bafd029ba8504cb0/qa/tasks/cephadm.py", line 1368, in wait_for_service while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (301) after waiting for 300 seconds 2024-03-12T03:09:07.029 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-12T03:09:07.040 INFO:tasks.cephadm:Teardown begin 2024-03-12T03:09:07.040 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T03:09:07.058 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T03:09:07.074 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-12T03:09:07.074 DEBUG:teuthology.orchestra.run.smithi153:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-12T03:09:07.110 DEBUG:teuthology.orchestra.run.smithi163:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-12T03:09:07.125 INFO:tasks.cephadm:Stopping all daemons... 2024-03-12T03:09:07.125 INFO:tasks.cephadm.mon.smithi153:Stopping mon.smithi153... 2024-03-12T03:09:07.125 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mon.smithi153 2024-03-12T03:09:07.322 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:09:07 smithi153 systemd[1]: Stopping Ceph mon.smithi153 for 5cbf0f4e-e015-11ee-95c8-87774f69a715... 2024-03-12T03:09:07.822 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:09:07 smithi153 bash[14340]: debug 2024-03-12T03:09:07.331+0000 7f41ab54f700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.smithi153 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-03-12T03:09:07.823 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 12 03:09:07 smithi153 bash[14340]: debug 2024-03-12T03:09:07.331+0000 7f41ab54f700 -1 mon.smithi153@0(leader) e2 *** Got Signal Terminated *** 2024-03-12T03:09:08.222 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mon.smithi153.service' 2024-03-12T03:09:08.278 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T03:09:08.278 INFO:tasks.cephadm.mon.smithi153:Stopped mon.smithi153 2024-03-12T03:09:08.278 INFO:tasks.cephadm.mon.smithi163:Stopping mon.smithi163... 2024-03-12T03:09:08.278 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mon.smithi163 2024-03-12T03:09:08.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:09:08 smithi163 systemd[1]: Stopping Ceph mon.smithi163 for 5cbf0f4e-e015-11ee-95c8-87774f69a715... 2024-03-12T03:09:08.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:09:08 smithi163 bash[22347]: debug 2024-03-12T03:09:08.465+0000 7f42c13db700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.smithi163 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-03-12T03:09:08.733 INFO:journalctl@ceph.mon.smithi163.smithi163.stdout:Mar 12 03:09:08 smithi163 bash[22347]: debug 2024-03-12T03:09:08.465+0000 7f42c13db700 -1 mon.smithi163@1(peon) e2 *** Got Signal Terminated *** 2024-03-12T03:09:09.599 DEBUG:teuthology.orchestra.run.smithi163:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5cbf0f4e-e015-11ee-95c8-87774f69a715@mon.smithi163.service' 2024-03-12T03:09:09.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-12T03:09:09.649 INFO:tasks.cephadm.mon.smithi163:Stopped mon.smithi163 2024-03-12T03:09:09.649 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-12T03:09:09.774 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T03:10:00.583 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-12T03:10:00.720 INFO:teuthology.orchestra.run.smithi163.stdout:Deleting cluster with fsid: 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T03:10:47.698 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T03:10:47.715 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-12T03:10:47.731 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-12T03:10:47.733 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248/remote/smithi153/crash 2024-03-12T03:10:47.734 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/crash -- . 2024-03-12T03:10:47.773 INFO:teuthology.orchestra.run.smithi153.stderr:tar: /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-12T03:10:47.774 INFO:teuthology.orchestra.run.smithi153.stderr:tar: Error is not recoverable: exiting now 2024-03-12T03:10:47.775 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248/remote/smithi163/crash 2024-03-12T03:10:47.776 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/crash -- . 2024-03-12T03:10:47.792 INFO:teuthology.orchestra.run.smithi163.stderr:tar: /var/lib/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-12T03:10:47.792 INFO:teuthology.orchestra.run.smithi163.stderr:tar: Error is not recoverable: exiting now 2024-03-12T03:10:47.794 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-12T03:10:47.794 DEBUG:teuthology.orchestra.run.smithi153:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-03-12T03:10:47.830 INFO:teuthology.orchestra.run.smithi153.stderr:grep: /var/log/ceph/5cbf0f4e-e015-11ee-95c8-87774f69a715/ceph.log: No such file or directory 2024-03-12T03:10:47.832 INFO:tasks.cephadm:Compressing logs... 2024-03-12T03:10:47.832 DEBUG:teuthology.orchestra.run.smithi153:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T03:10:47.877 DEBUG:teuthology.orchestra.run.smithi163:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T03:10:47.890 INFO:teuthology.orchestra.run.smithi153.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-03-12T03:10:47.893 INFO:teuthology.orchestra.run.smithi163.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-03-12T03:10:51.440 INFO:tasks.cephadm:Archiving logs... 2024-03-12T03:10:51.441 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248/remote/smithi153/log 2024-03-12T03:10:51.442 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-12T03:10:52.059 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248/remote/smithi163/log 2024-03-12T03:10:52.060 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-12T03:10:52.381 INFO:tasks.cephadm:Removing cluster... 2024-03-12T03:10:52.381 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 --force 2024-03-12T03:10:52.507 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T03:10:53.697 DEBUG:teuthology.orchestra.run.smithi163:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5cbf0f4e-e015-11ee-95c8-87774f69a715 --force 2024-03-12T03:10:53.827 INFO:teuthology.orchestra.run.smithi163.stdout:Deleting cluster with fsid: 5cbf0f4e-e015-11ee-95c8-87774f69a715 2024-03-12T03:10:55.021 INFO:tasks.cephadm:Removing cephadm ... 2024-03-12T03:10:55.021 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-12T03:10:55.028 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-12T03:10:55.035 INFO:tasks.cephadm:Teardown complete 2024-03-12T03:10:55.036 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-03-12T03:10:55.045 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi153:/dev/vg_nvme/lv_1... 2024-03-12T03:10:55.046 DEBUG:teuthology.orchestra.run.smithi153:> sudo nvme disconnect -n lv_1 2024-03-12T03:10:55.392 INFO:teuthology.orchestra.run.smithi153.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-03-12T03:10:55.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:10:55.394 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi153:/dev/vg_nvme/lv_2... 2024-03-12T03:10:55.394 DEBUG:teuthology.orchestra.run.smithi153:> sudo nvme disconnect -n lv_2 2024-03-12T03:10:55.672 INFO:teuthology.orchestra.run.smithi153.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-03-12T03:10:55.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:10:55.674 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi153:/dev/vg_nvme/lv_3... 2024-03-12T03:10:55.675 DEBUG:teuthology.orchestra.run.smithi153:> sudo nvme disconnect -n lv_3 2024-03-12T03:10:55.976 INFO:teuthology.orchestra.run.smithi153.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-03-12T03:10:55.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:10:55.978 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi153:/dev/vg_nvme/lv_4... 2024-03-12T03:10:55.978 DEBUG:teuthology.orchestra.run.smithi153:> sudo nvme disconnect -n lv_4 2024-03-12T03:10:56.276 INFO:teuthology.orchestra.run.smithi153.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-03-12T03:10:56.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:10:56.278 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-12T03:10:56.278 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/scratch_devs 2024-03-12T03:10:56.293 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi163:/dev/vg_nvme/lv_1... 2024-03-12T03:10:56.293 DEBUG:teuthology.orchestra.run.smithi163:> sudo nvme disconnect -n lv_1 2024-03-12T03:10:56.563 INFO:teuthology.orchestra.run.smithi163.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-03-12T03:10:56.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:10:56.565 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi163:/dev/vg_nvme/lv_2... 2024-03-12T03:10:56.566 DEBUG:teuthology.orchestra.run.smithi163:> sudo nvme disconnect -n lv_2 2024-03-12T03:10:56.871 INFO:teuthology.orchestra.run.smithi163.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-03-12T03:10:56.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:10:56.873 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi163:/dev/vg_nvme/lv_3... 2024-03-12T03:10:56.873 DEBUG:teuthology.orchestra.run.smithi163:> sudo nvme disconnect -n lv_3 2024-03-12T03:10:57.183 INFO:teuthology.orchestra.run.smithi163.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-03-12T03:10:57.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:10:57.185 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi163:/dev/vg_nvme/lv_4... 2024-03-12T03:10:57.185 DEBUG:teuthology.orchestra.run.smithi163:> sudo nvme disconnect -n lv_4 2024-03-12T03:10:57.459 INFO:teuthology.orchestra.run.smithi163.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-03-12T03:10:57.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:10:57.461 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-12T03:10:57.461 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/scratch_devs 2024-03-12T03:10:57.478 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-12T03:10:57.488 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-12T03:10:57.488 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T03:10:57.491 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-12T03:10:57.513 INFO:teuthology.orchestra.run.smithi153.stdout: remote refid st t when poll reach delay offset jitter 2024-03-12T03:10:57.514 INFO:teuthology.orchestra.run.smithi153.stdout:============================================================================== 2024-03-12T03:10:57.514 INFO:teuthology.orchestra.run.smithi153.stdout:*hv01.front.sepi 44.4.53.2 2 u 20 128 377 0.185 2.222 0.437 2024-03-12T03:10:57.514 INFO:teuthology.orchestra.run.smithi153.stdout:+hv02.front.sepi 63.231.80.2 3 u 83 128 377 0.175 3.123 0.541 2024-03-12T03:10:57.514 INFO:teuthology.orchestra.run.smithi153.stdout:+hv03.front.sepi 74.6.168.72 3 u 107 128 377 0.192 0.665 0.299 2024-03-12T03:10:57.514 INFO:teuthology.orchestra.run.smithi153.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-03-12T03:10:57.541 INFO:teuthology.orchestra.run.smithi163.stdout: remote refid st t when poll reach delay offset jitter 2024-03-12T03:10:57.542 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================== 2024-03-12T03:10:57.542 INFO:teuthology.orchestra.run.smithi163.stdout:*hv01.front.sepi 44.4.53.2 2 u 142 256 377 0.201 -1.012 0.269 2024-03-12T03:10:57.542 INFO:teuthology.orchestra.run.smithi163.stdout:+hv02.front.sepi 63.231.80.2 3 u 103 256 377 0.179 0.564 0.287 2024-03-12T03:10:57.542 INFO:teuthology.orchestra.run.smithi163.stdout:+hv03.front.sepi 74.6.168.72 3 u 88 256 377 0.158 -2.407 0.361 2024-03-12T03:10:57.542 INFO:teuthology.orchestra.run.smithi163.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-03-12T03:10:57.543 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-12T03:10:57.553 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-12T03:10:57.555 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-12T03:10:57.564 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-12T03:10:57.574 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-12T03:10:57.584 INFO:teuthology.task.internal:Duration was 4114.538009 seconds 2024-03-12T03:10:57.584 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-12T03:10:57.594 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-12T03:10:57.594 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-12T03:10:57.597 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-12T03:10:57.646 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-12T03:10:57.646 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2024-03-12T03:10:57.647 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-12T03:10:57.661 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2024-03-12T03:10:57.662 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-12T03:10:57.677 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-12T03:10:57.677 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T03:10:57.706 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-12T03:10:57.785 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-12T03:10:57.786 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-12T03:10:57.788 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-12T03:10:57.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-12T03:10:57.988 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-12T03:10:57.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-12T03:10:58.003 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-12T03:10:58.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-12T03:10:58.055 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-12T03:10:58.057 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-12T03:10:58.070 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2024-03-12T03:10:58.073 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2024-03-12T03:10:58.106 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-12T03:10:58.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:10:58.149 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-12T03:10:58.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-12T03:10:58.156 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-12T03:10:58.169 INFO:teuthology.task.internal:Transferring archived files... 2024-03-12T03:10:58.170 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248/remote/smithi153 2024-03-12T03:10:58.170 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-12T03:10:58.231 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248/remote/smithi163 2024-03-12T03:10:58.231 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-12T03:10:58.259 INFO:teuthology.task.internal:Removing archive directory... 2024-03-12T03:10:58.259 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-12T03:10:58.265 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-12T03:10:58.301 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-12T03:10:58.313 INFO:teuthology.task.internal:Not uploading archives. 2024-03-12T03:10:58.313 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-12T03:10:58.324 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-12T03:10:58.324 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-12T03:10:58.326 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-12T03:10:58.332 INFO:teuthology.orchestra.run.smithi153.stdout: 1048603 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 12 03:10 /home/ubuntu/cephtest 2024-03-12T03:10:58.349 INFO:teuthology.orchestra.run.smithi163.stdout: 1048601 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 12 03:10 /home/ubuntu/cephtest 2024-03-12T03:10:58.352 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-12T03:10:58.363 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-12T03:10:58.384 INFO:teuthology.nuke:Checking targets against current locks 2024-03-12T03:10:58.413 DEBUG:teuthology.nuke:shortname: smithi153 2024-03-12T03:10:58.414 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T03:10:58.440 DEBUG:teuthology.nuke:shortname: smithi163 2024-03-12T03:10:58.441 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-12T03:10:58.444 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 01:52:04.059181', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2024-03-12T03:10:58.467 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_22:08:03-orch-reef-distro-default-smithi/7592248', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-12 01:52:04.057746', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2024-03-12T03:10:58.497 INFO:teuthology.orchestra.console.smithi153:Power off 2024-03-12T03:10:58.497 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T03:10:58.520 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off 2024-03-12T03:10:58.521 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:10:58.535 INFO:teuthology.orchestra.console.smithi163:Power off 2024-03-12T03:10:58.536 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T03:10:58.546 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-03-12T03:10:58.559 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2024-03-12T03:10:58.559 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:10:58.581 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-03-12T03:11:02.547 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:11:02.582 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:11:02.658 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-03-12T03:11:02.693 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-03-12T03:11:06.661 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:11:06.694 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:11:06.772 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-03-12T03:11:06.805 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2024-03-12T03:11:06.905 INFO:teuthology.orchestra.console.smithi163:Power off completed 2024-03-12T03:11:10.774 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:11:10.884 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off 2024-03-12T03:11:10.985 INFO:teuthology.orchestra.console.smithi153:Power off completed 2024-03-12T03:11:11.089 INFO:teuthology.run:Summary data: description: orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/jaeger 3-final} duration: 4114.538008928299 failure_reason: reached maximum tries (301) after waiting for 300 seconds owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=dfc02bace98345b2b0255bc249418893 status: fail success: false 2024-03-12T03:11:11.089 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T03:11:11.172 INFO:teuthology.run:FAIL